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PRECISE  ANALYSIS  OF  AN  OPTIMAL  PERTURBATION 
ESTIMATION  AND  CONTROL  PROBLEM 


1.  INTRODUCTION 

There  is  considerable  interest  in  state-estimation  problems  in  which  a  state  variable  evolves 
according  to  linear  dynamics  with  additive  Gaussian  white  noise  and  in  which  linear  state  measurements 
are  available  which  are  also  corrupted  by  additive  Gaussian  white  noise.  There  is  also  interest  in  associ¬ 
ated  control  problems  in  which  the  control  enters  the  dynamics  linearly  and  in  which  the  objective  is  to 
find  a  feedback  control  law,  which  specifies  the  current  control  as  a  function  of  the  currently  available 
state  measurements,  that  minimizes  the  prior  expected  value  of  a  quadratic  performance  criterion. 
These  cases  are  important  partly  because  they  are  the  forms  resulting  from  first-order  descriptions  of 
noise-induced  perturbations  from  nominal  behavior  in  a  wider  class  of  state-estimation  and  optimal- 
control  problems. 

If  the  description  of  these  perturbations  is  carried  out  to  one  higher  order  of  accuracy,  the  effect  is 
typically  to  introduce  quadratic  terms  in  the  dynamics  and  state  measurements  and  cubic  terms  in  the 
performance  criterion  of  the  control  problem.  The  resulting  estimation  and  control  problems  can  often 
be  rescaled  so  that  the  state,  control,  and  measurement  perturbations  are  of  order  unity,  and  the 
coefficients  of  the  added  higher-degree  terms  become  the  relatively  small  quantities. 

A  formal  analysis  has  been  made  [1]  of  a  class  of  estimation  and  optimal-control  problems  with 
this  latter  type  of  structure.  The  results  had  the  formal  appearance  of  giving  approximations  to  the  con¬ 
ditional  probability  density  of  the  current  state,  given  the  currently  available  measurements,  for  the 
state  estimation  problem,  and  to  the  control  generated  by  an  optimal  control  law,  in  the  control  prob¬ 
lem,  which  were  accurate  to  first  order  in  the  small  coefficients.  Similar  formal  results  were  obtained  in 
Ref.  2  for  the  corresponding  "smoothing  problem,"  i.e.,  an  approximation  to  the  conditional  density  of 
the  current  state  given  future  as  well  as  past  measurements.  Approximations  of  this  sort  are  of  interest 
because  they  show  how  the  solutions  start  to  change  as  the  problems  begin  to  depart  from  the  familiar 
linear-quadratic-Gaussian  form.  And  if  such  an  estimation  or  control  problem  arises  from  the  sort  of 
higher-order  perturbation  analysis  described  above,  this  degree  of  accuracy  in  the  solution  is  all  that  is 
formally  consistent  with  that  of  the  problem  formulation  anyway.  Also,  the  first-order  approximation  to 
the  conditional  state  probability  density  in  all  cases  has  the  interesting  property  of  being  (at  least  in  for¬ 
mal  appearance)  the  first-order  Edgeworth  expansion  of  that  density.  Questions  naturally  arise  about 
the  validity  of  this  formalism,  however,  namely  as  to  the  exact  sense  of  the  first-order  accuracy  and  the 
range  of  conditions  under  which  this  accuracy  holds. 

Some  mathematically  precise  answers  to  such  questions  are  developed  in  this  report  for  the  sim¬ 
plest  nontrivial  case  of  a  scalar  discrete-time  problem  in  which  the  only  higher-degree  term  present  is  a 
product  of  the  state  and  noise  variables  in  the  dynamics  (state-dependent  process  noise).  In  particular, 
limits  are  established  on  the  error  in  the  first-order  formal  approximation  to  the  conditional  state  den¬ 
sity,  for  all  but  a  set  of  realizations  of  negligible  prior  probability,  which  are  sufficiently  strong  to 
guarantee  that  the  corresponding  formal  first-order  approximation  to  the  optimal  control  is  indeed  accu¬ 
rate  to  first  order  in  the  perturbation  parameters.  Furthermore,  the  number  of  epochs  in  the  estimation 
and  control  problems  can  go  to  infinity  as  the  perturbation  parameters  approach  zero,  white  the  validity 
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of  these  results  is  maintained.  In  order  to  establish  the  accuracy  of  the  optimal  control  approximation, 
it  was  found  necessary  to  establish  limits  on  both  the  size  and  the  fluctuation  of  the  error  in  the  condi¬ 
tional  state  density  approximation  for  the  corresponding  estimation  problem,  the  latter  limit  being  in 
the  form  of  modified  Lipschitz  conditions. 

Because  of  their  limited  scope,  these  results  are  more  exploratory  than  definitive.  The  hope  is 
that  they  can  serve  as  a  guide  to  what  sort  of  results  could  be  achieved  and  what  some  of  the 
phenomena  are  that  might  be  encountered  in  a  more  general  investigation.  Even  in  the  limited  context 
here,  rather  elaborate  constructions  seemed  to  be  required  in  the  analysis,  and  the  proofs  become 
highly  computational.  One  difficulty,  for  example,  is  that  the  conditional  state  probability  density  actu¬ 
ally  diverges  for  very  large  values  of  its  argument.  Presumably,  some  corresponding  pathology  would 
also  arise  in  the  continuous-time  version  of  this  state-estimation  problem.  Thus  it  might  be  useful  to 
devote  some  effort  to  developing  more  sophisticated  and  elegant  concepts  and  methods  for  dealing  with 
the  phenomena  encountered  here  before  proceeding  with  this  sort  of  mathematically  precise  analysis  in 
more  general  contexts. 


2.  THE  STATE-ESTIMATION  PROBLEM 

We  consider  a  discrete-time  state  estimation  problem  in  which  a  real,  scalar  state  variable  x 
evolves  according  to  the  transition  equation 

x/+ 1  -  fxk  +  u,  +  (1  +  ^/X,)*,;  I  -  0,  ....  N  -  1,  (1) 

where  f,  uh  and  i 1>,  are  known  parameters  and  the  w,  are  independent  zero-mean  normal  random  vari¬ 
ables  with  variance  var  (w,)  -  qh  At  each  epoch  except  i  —  0,  a  noisy  measurement  of  the  current  state 
is  received,  and 

z,  —  x,  +  «,;/-  1  ... ,  N;  (2) 

where  the  n,  are  independent  zero-mean  normal  random  variables,  independent  of  the  wh  such  that  var 
(n,)  —  r,.  The  initial  value  x0  of  the  state  is,  a  priori,  a  normal  random  variable,  independent  of  the  w, 
and  n,,  with  mean  x0  and  variance  v0.  It  is  assumed  that  there  are  positive  constants  F,  Q,  and  R  such 
that 

1  >  I//I  >  F\  i-0 . N  -  1; 

kl  <  QF*\  1-0 . N  -  1; 

R  ^  vq  $  Q\ 
and 

R  <  \r,\  <  Q\  i  -  1 . N. 

The  quantity  A,  defined  as 

A  -  max  {|<A/|:  /  —  0,  ...  ,  N  -  1}, 

is  treated  as  a  perturbation  variable  in  the  ensuing  analysis.  In  other  words,  for  fixed  values  of  the 
other  parameters  in  the  system  of  Eqs.  (1)  and  (2),  results  are  obtained  as  functions  of  A  for  all 
sequences  with  this  maximum  magnitude,  A  generally  being  considered  a  relatively  small  quantity.  A 
further  restriction  is  assumed  for  the  parameters  of  the  unperturbed  problem;  this  is  described  in  a  later 
section.  For  convenience,  we  also  denote  the  sequences 
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Z/  -  {zi . z,);  /-  1,  ... ,  N 


Y,  -  {«0.  •••.«/};  i  “  0 . N  -  1, 

with  Z0  denoting  the  empty  sequence. 

The  basic  objective  here  is  to  establish  error  bounds  for  certain  approximations  of  the  conditional 
probability  density  functions  of  the  state 

p(xM/Z,)\  i  -  0 . N  -  1 

and 

p(x,/Z,)\  i  -  1 . N. 

In  the  unperturbed  problem  corresponding  to  Eqs.  (1)  and  (2),  h  -  -  0;  i  -  0 . N  -  1;  and  it  is 

well  known  that  these  conditional  state  densities  are  all  normal.  Their  means  and  variances  are  given 
by  a  simple  case  of  the  standard  Kalman-Bucy  filter,  the  variance  equations  for  which  are  listed  here  for 
later  reference: 

Mi+i  “  f?P<  +  Qh  i  *  0,  ....  N  -  1  (variance  of  p(jc,+i/Z,));  (3) 

p,  -  -M,r<  /  -  1,  ...  ,  N\  p0  “  v0  (variance  of  p(x,/Z,)).  (4) 

Mi  +  r, 


For  nonzero  values  of  h,  however,  these  densities  do  not  even  exist  in  general  for  large  values  of  their 
arguments.  For  example,  it  is  shown  in  Appendix  A  that  if  i|r 0  and  q0  arc  nonzero,  then  pXl(l/<bo)  does 
not  exist,  in  the  sense  that 


Hence  it  is  only  reasonable  to  seek  approximations  to  such  probability  densities  within  wide  but 
bounded  ranges,  which  leads  to  the  somewhat  elaborate  approximation  concepts  considered  here. 


3.  ESTIMATION  EQUATION  SYSTEM  I  FROM  FIRST-ORDER  FORMAL  ANALYSIS 

A  formal  analysis  similar  to  that  described  in  Ref.  1  suggests  that  the  above  conditional  state  den¬ 
sities  can  be  approximated  to  first  order  in  A,  in  some  reasonable  sense,  as  those  corresponding  to 

P*,+r*U,.„  UIZl)  ~  11  +  3  ?I+I  {,i  "  3/)1  (5) 


(f/Z,) »  [1  +  4-  rjy  (t3  —  3r)J, 
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according  to  the  well-known  formula  for  the  transformation  of  probability  densities,  where  x'+l,  xh 
m,+1>  vh  and  t ),  are  the  functions  of  Z,  specified  by  the  following  equations,  called  Equation  Sys¬ 
tem  I  for  future  reference: 


*/*+i  -  fA  +  u,\ 

m,+ 1  -  f}  v,  +  q,  (1  +  <M/)2; 

.  l/?v,lvll  .  3»*,o +*,«!. 

r+  r 


X,  “  Xi  + 


m, 


m,  +  r, 


V,  ” 


i*i  ' i 
m,  +  r, 


1  + 


(Z/  -  X/*)  + 

yfm,  | 

"i  +  'i  \ 

/  “  */*)  Y 

{m,  +  r,)2  | 

r,i |  *  l(z,  -  X/*)2  -  (m,  +  r,)l;  x0  as  given; 
;  v0as  given; 


(7) 

(8) 

(9) 

(10) 
(11) 


and 


i»/ 


i"i  +  r< 


VI 


(12) 


3.1  Further  Restrictions  Assumed  for  the  Unperturbed  Problem 

In  terms  of  the  unperturbed  problem  parameters,  let 
b  -  *  max  (p/.  />/.  1) 

and 

a  -  -j  min  {*t/.  a) 


for  some  constant  k  >  1,  and  let  Gt(h,  k)  denote  the  set  of  sequence  pairs  (Z/,  y,-i)  for  which,  for  all 
(*o  •  •  •  • .  d»v-il  with  max  (I*, I)  -  h. 

Condition  1:  |i)y|,  |ij/|  <  3 Wb; 

Condition  2:  |vy|,  |my|  €  [a,  ft]; 

Condition  3:  \xj\,  |jfy|  <  In  (1/ A); 

and 


Condition  4: 


hzjUL  <  1 1„  a/** 

mj  +  rj 


for  J  -  1,  . . . ,  I  and  i  -  1, . . . ,  N.  It  is  further  assumed  that  the  unperturbed  problem  parameters  x0, 
vo,  (//,  <ljy  fy+ r,  J  -  0,  ....  N  -  1)  are  such  that  there  exist  a  k  >  1  giving  a  and  b  as  above  and  posi¬ 
tive  numbers  h and  c,  such  that  for  every  h  <  band  every  /  -  1, ... ,  N,  (Z/,  ,)  €  G,  (b,  k)  if 

max(|x0  -  x0U»/l,  l*vol>  ^_niax  ,  (lwyl.  I«/IH  <  cVln  (1/b) 
and 


max{|wy|:  J  -  0 . /  -  lj  <  c  In  (1/A). 
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In  this  case,  it  follows  easily  from  the  independence  of  xo,  the  Wj,  and  the  n;  and  from  the  inequality 


Pr(|x|  <  /}  <  ’»  1  >  0; 


for  a  standardized  normal  random  variable  x  that,  a  priori. 


(13) 


.2  U  2»J 


where 


««.«->.  «:i-i . 


(t2-  maxUo.  9o.  m**.  (ft.  r,)}, 

. N 


if  I14I  <  c  In  (l//r)  for  i  —  0,  . . . ,  N  —  1.  This  bound  is  significant  because,  even  if  the  number  of 
epochs  N  grows  in  perturbed  problems  with  decreasing  h  as 

N-Vln  (l/j»). 


so  that  lim  N  —  <»,  this  prior  probability,  and  hence  also  that  of  {Z,.  Ky_,)  €  G,  for  i  -  1,  ,  N, 

A— 0 

approaches  unity  in  the  limit  as  h  —  0. 

As  a  verification  that  the  ensuing  analysis  does  not  take  place  in  a  vacuum,  it  is  shown  in  Appen¬ 
dix  B  that  the  preceding  assumption  is  indeed  valid  for  the  specific  case  in  which 

x0-  0, 

vo"  1. 

fi  ~  1/2  1  /- 0 . N-  1. 

q,  -  ri+,  -  1  I 

with  k  «  12,  h  «■  e~5,  and  c  -  0.02. 

3.2  Lipschltz  Conditions 

For  future  use  it  is  helpful  to  define 


Vi  vi3/i  ;  1.  •• 

.  ,  AT 

(14) 

A,*  **  v'  ;  i  "  0,  . 

...  N; 

(15) 

which  are  first-order  approximations  for  one  half  the  third  central  moments  of  the  conditional  probabil¬ 
ity  distributions  of  x,  if,  respectively,  Z,-\  and  Z,  are  given.  Equation  System  1  can  also  be  expressed 
in  terms  of  these  variables  if  we  use  Eq.  (IS)  to  replace  ij*  by  A,*  in  Eqs.  (10)  and  (11)  and  replace 
Eqs.  (9)  and  (12)  by 

A,*+1  -  f?  A  +  if,  *«*,(!  +  <M,)  (16) 
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Lemma  l:  In  the  context  of  the  previous  sections  there  exists  an  A*  >  0  such  that  if  A  <  A*  and 
if,  for  any  /  -  0, . . . ,  N,  (Zh  y,_i)  and  (Z,,  l ) '  €  G,  differ  in  at  most  one  component,  whose  values 
are  denoted  respectively  by  p  andp',  then 

Is,  i(z„  -  s,i(z„  y,_,)']|  <  r,  |P  -  P*|.  (18) 

I  k,I(z„  y,_,)]  -  v,kz„  y,_,)'il  <  r,  Alp  -  P'l,  (19) 

and 

lx,  uz„  y,_,)j  -  x,[(z„  <  r,  a2  |p  -  P1,  (20) 

where  T,  is  defined  by  the  recursion 

r«  - 1  +  T  +  (|! + i|" +  50' +  ^  +  f  (J + f  | +  ^(' +  30  +  f  )| r- 

A-0 . N  —  I;  r0-0. 


Proof-.  The  conclusion  is  trivially  true  for  /  —  0.  Assume  it  is  true  for  epoch  i.  Then  it  is  pri¬ 
marily  a  matter  of  computation  to  express  S/+,,  vi+u  and  A/+1  as  explicit  functions  of  S,,  vh  A,,  and 
2,+i  via  Eqs.  (7),  (8),  (10),  (11),  (16),  and  (17),  obtain  the  partial  derivatives,  apply  the  results  in 
Appendix  C  for  Lipschitz  conditions  of  composite  functions,  and  use  the  inequalities  involved  in  the 
definition  of  Gl+i  and  the  fact  that  the  definitions  of  <7,  and  G,+)  imply  that  every  initial  segment  of  a 
sequence  pair  in  G,+i  at  epoch  /  is  an  element  of  G,  to  show  that  inequalities  (18)  to  (20)  hold  under 
the  conditions  of  Lemma  1  for  epoch  i+l  with 

rM"l+7?f+j|2+^'|<l/:|,+3®>'Hi£‘+fr(l  + 1/1 + 3^| 

+irk+3*+£llr- 


at  least  if  A  is  less  than  each  of  some  finite  set  of  positive  values  which  are  needed  to  imply  various 
computational  inequalities  but  are  independent  of  i.  Using  the  inequalities  for  |q,|,  |/,|,  and  |r/+1|  in 
the  problem  formulation,  we  can  then  show  that  the  conclusion  of  Lemma  1  holds  at  epoch  f+1  for  A* 
equal  to  the  minimum  of  this  finite  set  and  A  of  the  preceding  section,  which  is  a  positive  constant 
independent  of  /.  Hence,  Lemma  1  follows  by  induction  on  /.  □ 

4.  APPROXIMATION  OF  CONDITIONAL  STATE  DENSITY 

In  order  to  develop  an  induction  argument,  we  next  consider  a  generic  epoch  i  ^  N  —  1,  delete 
the  subscripts  of 

A,,  Xf,  Uf ,  ft  i A,»  x,,  v,,  2j+i,  ft c,,  D,,  Af/, 
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the  last  four  to  be  defined  shortly,  suppress  conditioning  on  Z,  in  the  rotation,  and  denote 


a(/)  —  yfhr  e  2  !/>*_*  (/)  -  px-i  (f)|, 

1  7T  7T  1 

if  the  density  px_i  exists  at  t. 

4.1  Assumptions  at  Epoch  i 

For  the  purposes  of  Sections  4.2  and  4.3,  the  following  four  conditions  are  assumed  for  epoch  / : 

Assumption  1:  For  all  t  such  that  |f|  <  M,  where  ~-h~'n  <  M  <  -j -h~u\  p  ,  .  (r)  exists  (at  least 

4  4 

V  V 

as  a  Radon-Nikodym  derivative  of  the  probability  distribution  function  of  which  itself  is  clearly 

V  v 

well-defined  since  only  Borel-measurable  functions  of  Gaussian  random  variables  ever  appear  here), 
and 

l«(f)|  <  (lh2eir\  where  ft  <  h~UA. 


Assumption  2:  Pt 


X  -  X 

V7 


~l,v*  , 

|  >  t  <  ke  2  ,  k  —  2 1  +  1,  for  all  t  > 

O 


Assumption  3:  (Z,,  F(_()  €  G,  and  (Z,  •  z,  F,_ t  *  u)  €  C,+|,  where  'denotes  concatenation. 

Assumption  4:  If  (Z,.  F/_])'  €  G,  and  differs  from  (Z/,  Y,- \ )  in  at  most  one  component,  whose 
values  are  denoted  respectively  by  p'  and  p,  then 

|a(r,  p)  -  a(r,  pOl  <  e21'1  (CTt3  +  Dh2\p  -  p'l)  if  |r|  <  M 

and 

|a(r,  p)  -  a(s,  p)|  <  **"**IW.I*II  (ca3  +  Dh2\s  —  / 1)  if  |s|,  |r|  <  M, 
where  C  <  h~l/*  and  D  <  h~v*. 

4.2  Linear  Transformation 


Applying  the  well-known  formulas  for  the  transformation  of  the  probability  density  for  a  function 

of  a  random  variable  to  — )=■  (y  —  y),  where 
vm 

y  -  fx  +  u. 
y  -  fi  +  u, 


W.W  WILLMAN 


and 


m  -  f2  v, 

we  can  show  that  its  density  exists  at  /  for  I  / 1  <  M  and  that  there 


Pv-i  0)  “  Px-i  0). 

#  7T 


Hence, 


a(t)~ 


>/2w  e  1  P^-i  (f)  ~ 
Vm 


l  +  |t,(rJ-r) 


A  a(r). 


Lemma  2:  In  the  preceding  context,  Assumptions  1,  2,  and  4  of  Section  4.1  hold  with  x,  x,  v,  «, 
and  Y,-\  replaced  respectively  by  y,  y,  v,  a,  and  Y<-i  *  u,  where  (Zh  J/_ i  •  «)'  must  be  such  that  (Z,, 

Y,-iY  €  <?,. 

Proof.  Assumptions  1  and  2  are  established  by  the  preceding  remark  and  a  few  simple  algebraic 
manipulations.  Assumption  4  follows  from  the  fact  that  a  is  independent  of  u,  from  Assumption  4  for 
a,  and  from  the  fact  that  a  -  a. 


4.3  Addition  of  Process  Noise 


Now  let 


y  -  x  +  (1  +  4>X )  w;  w  -  — 


and  define  the  random  variable 


Zj-Z. 

vm 


where 


m 


v  +  *(l+0*)2;  «-4. 


// 


f 


Then,  since  I  <  1  by  the  assumptions  of  Section  4.1, 


+  l+*»  l  + 

v  m  1  •+• 


where  to  is  a  normal  (0,1)  random  variable  and 

_jL 

,  i*2 


pU) 


/2ire2 


1  +  ^  (r3-  3«2f)  +  a  - 
3«3  « 


for  |/|  <  «  M,  where  t  -  ■*/— .  Let 

v  w 

<r  -  -v  U  +  4>x)2, 


(21) 


(22) 


and 


r 
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I tty/m 
1  +  (i/X 


<u. 


g  -  a  r. 


(23) 

(24) 


L-eM. 

At  this  point,  we  also  assume  that  q>  &  0,  so  a  >  0.  If  q,  «  0,  Lemmas  3  to  5,  to  be  proved  later  in 
this  section,  are  trivially  true. 


Since  s  -  t  +  g,  it  follows  that  for  any  A  >  0 
Pr{s€  [0,  9  +  A))  -  Pr{(r  +  g)  €[0,  9  +  A)  and  |/|  >  L\ 

+  Pr{(f  +  g)  €  19.9  +  A)  and  |/|  <  I). 

We  partition  the  first  event  in  Eq.  (25)  at 

oo 

U  (£*  U  Fk). 

k-0 

where 

Ek  -  [l.g:  U  +  g)  €  1 9.9  +  A)  and  /€  ( -L  -  (k  +  1)  A,  -  L  -  *A)} 
and 

Fk  -  k*:  (t  +  g)  €  [9.9  +  A)  and  f€  [L  +  *A,  L  +  (ft  +  1)A)). 

Now, 


Ek  C  {r,g:  r  €  A.  g  €  B\. 

where 

A  -  (-L  -  (k  +  D  A,  — L  -  *A] 
and 

B  -  [0  +  L  +  fcA,  9  +  L  +  {k  +  2)  A), 


so 


Pr{£*)  <  Pr{g  €  B/t  €  a)  •  Pr{r  €  A). 

For  nonzero  f ,  the  density  p, /,(.()  exists  such  that 

£2 


p*/M)  - 


e  2<r2(,+*')J  ,r.  ^  1  j  4  ifrVm  . 

VW(1  +7D1 

0  if  r  -  4"- 


For  a  given  this  density  is  maximized  over  all  t  if 

<r(l  +  4>t)2  “  f2, 

in  which  case 

1 


P*/M) 


l€  \y!2ire  ' 


(25) 
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Hence,  since  all  the  events  and  probability  measures  are  well-defined, 
Pr{*€5//€,4}  -  f  P r{gtB/t)  dPit/ttA) 

**  f*A 

<  f  (2A  max p./,(x)]  dP(t/t€A) 

Jit  A  KH 


'  Jit  A - (tB  -  - 

<  f  ~—=dP(t/t€A)  if  0  >  -L  +  1,  by  construction  of  B 

JitA  dive 


4  vstX,,  dPUI^A)-jt; 


Therefore, 

Similarly, 


Pr{£j  < 


■Jive 


Pr[t€A)  if  0  >  -L  +  1. 


Pr(Fj  <  -4^  Pr{/€[I  +  A:A,  L  +  (k  +  I) A)}  if  0  <  F  -  1. 
dine 

For  |0|  <  L  —  1,  it  follows  from  the  definitions  of  Ek  and  Fk  that 
Pr{5  €lM  +  A)  and  |r|  >  L)  -  Pr{  U  ( Ek  U  F*)) 

k-0 

-  £  [Pr{£*}  +  Pr{Fj] 

*-o 

<  f  (Pr{/€  (-1  -  (it  +  1)  A,  -L  -  ArA]} 
dine  £r0 

+  Pr{f€U.  +  *A.  L  +  (fc  +  l)A))] 

<-iir[Pr{f€  U  (~L~(k  +  1)A,  -L-*A]} 
dine  k- o 

+  Pr{r€  U  [L+  fcA,  L+  (A:  +  1)A)J] 

*-o 

(since  the  Ar-indexed  intervals  are  all  disjoint) 
2A 

—  -j==-  Pr{|/|  >  L)  by  construction 
dive 

<  ±  Pr{|ri  >  L). 

It  is  shown  in  Appendix  D  that  this  inequality  implies  that  the  measure  on  the  real  line  induced  by  the 
quasi-distribution  function 

F,(0,L)  *  Pr{s  <  0  and  |r|  >  L ) 

is  absolutely  continuous _with  respect  to  Lebesgue  measure,  so  that  FS(9,L)  has  at  least  a  Radon- 
Nikodym  derivative ps(9,L),  and  such  a  derivative  exists  with 

0  </>,(*,!)  <  Pr{|r|  >  L } 

for  |0|  <  L  -  1.  Using  the  usual  abbreviations  for  probability  density  notation,  we  have 

_£ 

pis, l )  <  Pr{ | f  |  >  L)  <  lSlh'yfe  <PiM  -  I)  +«*>(A/)  +  d(,  (26) 
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where  <t>  denotes  the  tail  of  the  standard  normal  distribution,  since 


1 1\>  L  **  *-j=£  >  M. 


It  is  well  known  that  for  M  >  0 


Jl^  di  <  If  "jUr  ’ 

and  it  follows  from  repeated  integration  by  parts  that 

ti  m2 

jM  ?  e  2  dt  -  (M2  +  2)  e  2  . 

Using  these  inequalities  in  Eq.  (26),  we  can  show  that 


p(s,L) 


e  2  1  2ftA2  ew+  1 

■J2n  l 

M-  1 

e  1 

1  2(1  h2 

Jin 

\  M-  1 

-4+w 

e  1 

[  211  A2  1  - 

+  \M2  +  2  + 


j  y(s2-«J)-|j| 


m-  i  r 


+  A/2  +  2  + 


1  j 


M  —  1 


With  M  >  \  h  l/l,  it  follows  for  sufficiently  small  positive  h  and  for  Is  |  <  eM  -  6Vln(l /h)  that 
4 


-4r+l*l 


p(s.L )  <  e  J—  [nAVl,n<,/*)  +  ( M 2  +  3)  e-*,n,,/*)] 
-■^+1*1 

<  [nA,0+  A7(l  +  3A)J,  since  M  <  h~l/*. 


Since  t  has  a  probability  density  function  (in  the  sense  of  a  Radon-Nikodym  derivative  of  its  dis¬ 
tribution  function)  by  assumption  for  arguments  less  in  magnitude  than  L,  the  second  event  in  Eq. 
(25)  can  be  evaluated  as 

r  ^ 

A  J  s_L  p,  (s  -  a-r)  prh  ( r ,  s  -  trr)  dk„ 

c r 

where  X,  denotes  Lebesgue  measure  for  the  r  variable,  since  s  -  t  +crr  and  the  conditional  density  of  r 
given  t  is  well-defined.  Hence  the  results  of  Appendix  D  also  show  that  the  quasi-distribution  function 

Fs  ( 9,L )  -  Pr  {s  <  0  and  |/|  <  L ) 

has  a  Radon-Nikodyn  derivative  ps  (9,  L)  such  that 

r 

p(s,L)  -  J  s_L  p,  (s  -  <rr)  pr/l  (r,  s  -  <rr)  dkr  (28) 


11 


W.W  WILLMAN 


and  therefore  that  s  itself  has  a  probability  density  in  this  sense  (for  arguments  of  magnitude  less  than 
L- 1)  such  that 

P,  (0)  “  Pi  (0.  L)  +  ps  i 9,L ) 

or,  in  abbreviated  notation, 

pis)  -  pis,L )  +  pis,L). 

The  conditional  density  for  r  in  Eq.  (28)  is 


,M0~  Mu 


To  approximate  this  density  in  Eq.  (28),  denote 

,2 


fix ) 


e  2«  +  *>J 

(or  zero  if  x  -  -  1), 


V2ir(l  +  x)2 

where  x  now  just  denotes  a  real  variable.  Then,  for  r  ^  0, 

/*(*>  ■  [|t£ — F  - , 

1  +  X  1 1  +  xj 

and 

/"(*)  -  _/kl 


(or  zero  if  x  -  -  1) 


(29) 


(i + x?  u  ifi|  - 5  |ttt|  +  2|  (or  “ro  if  * "  -  °- 

For  r  i*  0,  these  derivatives  are  continuous  for  all  x,  so  Taylor’s  theorem  with  remainder  implies  that 

fix)  -  /(0)  +  rfix)  +  \  x2  /"(»). 

for  some  9  in  the  union  of  intervals  [x,0]  U  (0,x),  where  [0,x],  is  regarded  as  empty  if  x  <  0,  etc. 
For  r  -  0,  this  argument  still  applies  if  |x|  <  1. 

Therefore,  for  |s|  <  using  0(s  -  err)  to  play  the  role  of  x  we  get 


Prh  ir,  s  -  err)  -  U  +  0  is  -  err)  ir1  -  D) 


r* 

1  A2  f  s-<rr 

2 

e  2tl  +  4  (i  -  <r»)lJ 

2*  [l  +0(s  -<r9) 

y/2nll  +  0(s  —  <r9)]2 

rr  * 

f 

l2  1 

1  r  5 

||  1  +  0(s  -  <r9) 

1 1  +  0(i  —  <r9)  j  )’ 

where  9  now  denotes  some  value  in 
theorem  for  a  first-order  expansion  of  fix 


J 

■fl”H 

fix)  likewise  i 


(30) 


which  might  vary  with  r  and  s.  Applying  Taylor's 
gives 


pr/l  ir,  s  ~  <jr)  ■*  -  +  0 


s  -  err 


1  +  0  is  —  w0) 
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2(1  +  a<»  -  «■•>!* 


>/2wll  +  4  (s  -  <r 0))2  II  1  +  ♦  (*  -  <r0) 


-  1 


(31) 


for  | s |  < 


L  where  the  value  of  0  is  generally  distinct  from  that  of  Eq.  (30)  but  in  the  same  range. 


Substituting  Eqs.  (22),  (30),  and  (31)  in  Eq.  (28),  and  changing  the  variable  of  integration  to  u  — 

s  —  err,  gives  the  following  for  Is  I  <  l-j-l: 

a 


si 

2«2 


p(s.L)-( 

J~L  yJlir*2 


{>  -  n)2 
2a  2 


•Jlvar1 


l  + 


du 


fL  J- 
J-L  J 


u1 


V2ir«2 


(i  -  a)2 

e  2a2<l+40)2 


•s/2ircr2  (1  +  +0)* 


1  +  .  J.2.,  1^1 

V  |  2 


s  —  u 


<r(l  +  40) 


-  5 


s  —  u 


<r(l  +  40) 


4*  2f  </u  + 


« (f 


Jil 

2.2 


V2w? 


(»  -  a)2 
2<r^ 


+  4« 


(s  -  u)2 

e  2a2  <1  ♦  4r>2 


y/iita1  (1  +  4y)2 


s  -  u 


«r  (1  +  4y) 


-  1 


</Xu, 


(32) 


where  j8  and  y  €  [«,  0]  U  [0,  u]  and  vary  with  u  and  s.  Completing  squares  in  the  exponents  of  Eq. 
(32)  and  using  the  fact  that  <r2  +  «2  —  1  by  definition  gives,  after  some  manipulation, 

•2 


p(s,L ) 


yfht 


*1 

I,  +  ~*2l2  +  E, 


where 


<>  -  «2»)2 


/|  -  f  g  , —  4-  1  + (u3  -  3«2u)  l+4«  - — -  -1  du. 

J~L  V2wcr2€2  3«3  o- 


(33) 


(34) 


W _ 

rL  g  2«2  2a2  (I  +  4S)2  [  -  . 

I  /■  t  =  =;•  ll  +  “V  (u3  —  3f2«) 

,'-t  V2i r«2  V2irc2  (1  +  40)*  [  3«3 


(»  -  «)2 


s  -  u 


<r  (1  +  40) 


-  5 


s  -  u 


<T  (1  +  *0) 


+  2  u2du, 


(35) 
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and 


u)1 


it 


sfi 


+  0« 


<5  -  H>* 
2<rJ(l  +  *y>} 


■no 


ylino1  (1  +  0y)* 


s  -  u 
a(l  +  4>y) 


dku.  (36) 


Approximation  of  l\ 

Let  t  now  denote  a  new  variable  of  integration  in  Eq.  (34)  such  that 


a« 


Then  u  —  e  ( art  +  es),  — — —  —  as  4-  e/,  and 

c r 

.  4 — U  2 

1\  —  J _€,  Tg~ll  +  Co  +  +  fj/3  +  c4f4  +  csrs  +  ctf6l</t,  (37) 

<T 

where 

Co- «J|-^-+^a2|s3— «(t|+^<)s  +  '^^-(«2aJs6—  (l  +  2a2)s4  +  3s2J, 

ci  —  «aWjc  +  (3a2-2)0]s2-a(Tj+0€)-  1 (6a4- 8a2+2)s5  +  (8a2-2)s3-6s], 

Cj  —  [i}<a2+0g2(l  — 3a2)ls  +  l«2(15a4— 10a2+ l)s4— 3(2a2  —  l)2s2  +  3a2], 

C3  —  y  (ij<r2  +  30«2)  +  y0i}a€2I(lQa4  —  10a2  +  2)s3  +  (4a2  —  3)s], 
c4  —  y  d»Tj«a2[(15a4  —  20a2  +  3)s2  +  2a2  —  3], 
c$  —  y  ^ija3«2( 2  -  3a2) s,  and 
c6  -  y  *rja4«2- 


We  now  restrict  5  so  that  |s|  <  L  also,  which  implies  |cs|  <  L,  so  applying  standard  results  for 
moments  of  normal  densities  to  Eq.  (37)  we  obtain 

/1  -  1  +  y  (tj«  +  30a2)  (s3  -  3s)  +  -  R2,  (38) 

where 

R,  -  [a2e2(s‘  -  12s4  +  27s2  -  6)  +  3(1  -  3a2)s2] 
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*‘~L  + 


and  where  A  denotes  U  |  ^  —*-5 ,  00 J 

Since  0<<r<l,  0<e<l,  and  cr2  +  «2  -  1  by  their  definitions,  it  is  easy  to  show  that  <r2«2  <  1/4 
and,  after  some  manipulation,  that 


l*i!  < 


\±HL 

12 


(s  +  2)6 


for  any  s.  Applying  inequality  (E2)  of  Appendix  E  then  gives 


!*il  < 


<t>W 


12 


e,j|  for  all  s. 


Using  the  definition  of  4>  and  the  bounds  assumed  for  v,  Jr,  and  m,+i,  we  get 

»lil4 


and 


Denoting  --  -  --  by  t  and  — — —  by  fi  for  convenience,  we  have  for  all  (  >  0 
<r  cr 

|  $  I  <  L  —  t  11  >  L  +  >  0 

f  a 


r<~— L-— £<0. 
«  er 


Hence,  integrating  by  parts  gives 
l 


(1  +  Co  ■+■  c2  +  3c4  +  15  c$) 


(xie  2  dt+IM  e  1  d\ 


7^ 


+  (ci  +  2c3  +  8c5)|e  2  -  e  2  I  +  (c2  +  3c4  +  15c6)Le  2  -  re  2 


-el  -4  I 


+  (cj  +  4cs) 


H2e  2  —  t2 e  2  I  +  (c4  +  5c6)|m3c  2  -r3e  2 


-el 


+  cj  2  -  r*e  2 

+  C6 

^i5e  2  -  r5t>  2 

Denoting  —£.  +  —{  by  8,  we  obtain  the  bound 
c  cr 

B 2 


,**1  <  737 


2c6  B5  +  c5  B*  +  2  (c4  +  5c6)S5  +  ft  j  +  4c5)B2 


(39) 


(40) 
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4-2  (c2  +  3c4  4-  15cj)B  4-  — (1  4-  Co  4-  c2  4-  3c4  4-  15c$) 


since 


.0.  _ii  ,  _iti 

e  2  du  <  —  e  2  . 


Since  only  values  of  B  that  are  large  compared  to  In  (1/A)  will  be  of  concern  here,  it  follows  from  the 
sizes  of  c0  through  c6  that  this  last  bound  for  |B2|  varies  essentially  as 

2_  e  2 
7T  B 

for  sufficiently  small  positive  A.  Since 

_ai 

2?  >  yJin(i/h)  -►  ■>/£  <  757 for  *  <  7- 

then 


iieui 


(41) 


for  B  >  3  Vln(l/A)  and  A  <  — .  This  bound  on  B  is  implied  by  the  condition 

e 

Is  1  <  L  -  6>/ln(l/A), 

because  if  —  <  then  B  >  —L  (of  order  A-1/4);  otherwise  B  >  -i(L  -  |s|). 

(Til  2 

Applying  the  triangle  inequality  and  Eqs.  (38),  (40),  and  (41),  we  get 


|/i  —  1 1  4"  y(*»«  4"  34><r1)(si  -  3s) 


(42) 


for  sufficiently  small  positive  A. 
Bound  on  |/2| 


For  A  <  b~ 4,  A/  <  -  and  |x|  <  -^  by  assumption.  Since  L  <  M,  it  follows  that 


w  <  ^  * 


and 


|u|  <  L  -►  ld>u|  <  y-v/A 

under  these  conditions.  Hence  for  sufficiently  small  positive  A, 

.  (»  -  k)j 

(S-U)2  - 


2*2(l  +  *0)2 


2<r2(l  +  -jV*)2 


VW(l  +  *4)>  <  ^2^’d  - 


(43) 
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if  (i  €  (m,0)  U  (0,m),  and  from  Eq.  (35), 


<j  -  u)2 

If 2 

/  e  _2,J 

ir2(  1  +  J-v/J)2 

<?  3 

1  +  -^(u3-  3m) 
3« 

-t  y/2nt2 

2rr(72(l  -  -J-s/A  )2 

2|i^“r  +  6irj'|1+2|  «’*. 

<J  |  <T 


(44) 


Completing  the  square  in  the  exponent  of  Eq.  (44)  and  using  the  fact  that  t2  +  a7  -  1,  we  get 


l/jl  < 


i  +  }Va 

1— y>/A 


7 


20  +  y ir^fh  4*  y<r*/r) 

2ir(l  +  ytrV/f  +  y<r2A) 


(u-u)2 


where 


«2s 


1  + jtrVA  2+ j>/A 


and  v  «■ 


'  +6[i_" 

2 

+  2 

M2  du. 

«V2  jl  +  -jVaJ 

1  +  j<r2>/A 

2  +  ■ 

Now,  for  s  >  0  and  k  >  0, 

6 


s  < 


o- Va  (2+yVh ) 


+  2 


k  s  < 


2fc|3+0-Vh|2+-jVhj 


o-Va|2+-j>/A 


(45) 


1  - 


I  +  ytr2>/A  2  +  ^p/A 
3  3  j 


<  2k 


1+-5-ct2VA 

2+-U/A 

3 

3 

<  ~  y  +  ks. 


W.W.  WILLMAN 


Since  cr2  <  1, 


^  2  1  +  yo-VA  2  +  jyfh 


<- Y  +  *UI 


for  both  positive  and  negative  s  for  all  sufficiently  small  positive  A.  Using  this  result  for  k  —  1/8  in  Eq. 
(45)  leads  to  the  bound 


2«2 

e  2 

>/2tr 

for  sufficiently  small  positive  h,  since 

Since  It)  I  <  3  hJb  by  assumption,  the  integral  in  Eq.  (46)  is  bounded  by 

by  the  triangle  inequality  and  elementary  properties  of  integrals.  Changing  the  variable  of  integration  in 
(46)  to 

„  ii—5 
o-eV 2 

leads  to  the  inequality 

_ii+M  _ii  /  » 

W  <  *  ^Jo  75^-{i  +  A>/56[('+l*l)3+3(f+UI)]| 

x  [8(r  +  |a|)4  +  12(/  + l*l)J  +  2](r  + Isl)2  di  (47) 

for  sufficiently  small  positive  A,  since  er  and  c  <  1,  and  since 

-*-V5<rr  + - *4 - r 

1  +  -~-o,2>/A  2+y  VA I 

and 

o'!  +  -r>/A  s 

s-u  «  .  .  l  3  _ 


^-V2e?  + 

<7 


1  +  ycrVA  2  +  yVA 
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SO 


_w 

( 


<  s/5 (1/1  + III) 


and 


s-u 

a 


<  s/2(U|  +  |s|) 


for  sufficiently  small  positive  h.  Also,  for  sufficiently  small  positive  h ,  the  bound  of  (47)  is  smaller 
than 


l512(/6  +  s&)  +  19204  +  s4)  +  8 02  +  s2)]  di 


by  inequality  (E2)  of  Appendix  E,  and  by  inequality  (El)  of  that  appendix  and  standard  results  for  nor¬ 
mal  moments,  the  integral  in  this  expression  is  less  than 

256s6  +  96s4  +  4s2  +  4132  <  256(|sl  +  2)6  <  —■ ?6)-(?6)-  e''*'. 

e* 

Substituting  back  into  inequality  (47)  and  using  the  definition  of  |^|,  we  get 


■y  d>2  h 

< 

[»|(W2J  «S6KU764<»  4 

9  J  e 4 

e'2 

yfllT 

(48) 


for  sufficiently  small  positive  h. 
Bound  on  E 

From  Eq.  (36) 


U 1 

(s-u)1 

f-‘« 

u|  e  2,2 

e  *2 

*  j  -Jlitf.1 

oJlntr2 

£-(?)  i 


u‘ 
It 1 


V2ir«^ 


d\u 

(s-u)1 


e  te’d+ty)2 


>/2ir(r2(l+$y)2 


s-u 


o-(l+^y) 


-  1 


u  d\u. 


(49) 


By  assumption  |a|jj|  <  fth2e]u/^  for  |«|  <  L.  By  the  triangle  inequality  for  integrals,  the  first  of  the 
two  terms  in  Eq.  (49)  is  bounded  in  magnitude  by 


u1 

(|-u)J 

fL  >/'  1 

e  2,2 

J-L  e 

yJ2nt2 

'JltrtA 

du. 


Completing  the  square  in  the  exponent  gives  this  bound  as 

t1  tr 
2  ~  2 


Sih2 


v2 n 


(u-t^s-a^t)1  (u-t1s+<r1t)1 

/0  p  2a1*1  SI  pi  2tr1t1 

.  - <  i  g —  du  +  e  2  J  - j  '~k  > —  du  . 

-L  ./■> - 2.2  Jo  y/2nPP 


-0 


_Jti 

(s-u)1 

f*  aUl-C^ 

e 

dku 

£  2  Clh2-?* 


r<n 


W.W.  WILIMAN 

The  reasoning  used  to  establish  inequality  (41)  likewise  shows  that 

(*-«>*  (*-«>* 
e  IrJd+^y)1  g  fe20+l/W*)> 


^2wtr^0-^y)^ 


lit  <T* 


(51) 


1+4 -yfhl 


for  sufficiently  small  positive  h.  Hence,  from  the  triangle  inequality  for  integrals,  the  second  term  in 
Eq.  (49)  is  bounded  in  magnitude  by 


»**■}=# 


jr 

V 


•Jin*1 


(i-m)2 


2*jo+|v£>j 


2ir<r2(l  +  -|->/A  )2 


s-u 


-1 


udu. 


(52) 


Completing  the  square  in  the  exponent  gives  the  exponential  factor  in  the  integrand  of  Eq.  (52)  as 


V? 


[»  -  «  «tn(n))2 


1(1  +  4-ff  (7  +  -W*>1 


(«  -  i>* 


(53) 


where 


and 


u  -  sgn  (u)  + - '•>‘+.*.(‘‘>1 

1  +  —  <r2  ->/h  (2  + 


yfh ) 


€2<r2(l  +  |VA)2 
1  +  ^<riyfh  (2  +  -jVA  ) 

The  first  factor  in  brackets  in  Eq.  (53)  is  bounded  by 

.  rr  <1*1  + 

1  +  vft  -2(1+  VI) 

for  sufficiently  small  positive  h.  This  bound  is  equal  to 

}*z+l*l 


,wa 


l+^/TTTr  orl+^L-^+iJiLI2 

v27re  v2ire  *  f 

Since  a2  <  by  assumption,  the  first  two  factors  in  Eq.  (53)  are  bounded  uniformly  in  u  by 


eXi  (1  +Vh) 


V2ir 


,1*1 
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for  sufficiently  small  positive  h.  The  last  factor  in  Eq.  (S3)  is  bounded  by  the  sum 

(M  -  6t)*  (■/  -  &;)* 


where 


A  a 

«(--€  + 


_ €2(s  +  <) _ 

1  +  yo,2>/A  (2  +  y>//f  ) 


«  a 

u2  -  e  + 


_ t2  (s  -  c) _ 

1  +  y  (Ti'J~h  (2  +  y  Vh  ) 


Hence,  the  second  term  in  Eq.  (49)  is  bounded  in  magnitude  by 


.  r- . ,  1  ~V+I,I  “ 

2!lh,  ilAjhJi  tTi  e_>  t  L 

1  -  y/h 


£ 


-  \\udu 


s.y 


(u-ii2)2 

2* 


“  V2iri» 


For  either  of  the  two  integrals  in  Eq.  (54),  let 


1  u  du\. 


u  -  u  __  s  -  «  s  -  fl  .  Vp  . 
i  —  — *—  ,  so - - - H - r. 

Vp  a  a-  a 


-pffi  (2  +  p/J) 

«-T«<r  1  +  - : - : -  +  - : - J - , 

1  +  y  <r2-Jh  (2  +  yV/T  )  1  +  —<T2yfh  (2  +  y -Jh  ) 


«2a  (1  +  ~r-Jh  )2  1  +  g  j— ^  Vft  (2  +  4-  -Jh  ) 

5  -  u  _ _ 3 _ t  ±  ta  _ - _ 3 

°  1  +  y  <r2Vh  (2  +  yV/i  )  1  +  y  <r2'J~ti  (2  +  y>/^ ) 

a2  (1  +  \>[}\  (2  +  -rVft  )] 

+  J.  _ 3 - 3 -  , 

^  1  +  y  <r2V)f  (2  +  y-\/7j) 

so  that,  for  sufficiently  small  positive  h, 

|u|  ■  |Vp  t  +  u|  <  2(|s|  +  |r  |)  +  l 


21 


and 


W.W.  W1LLMAN 


<  2  (( s I  +  If!)  +  1. 


Thus,  each  of  the  two  integrals  in  Eq.  (54)  is  bounded  in  magnitude  for  such  h  by 


l8(|s  |  +  If  I)3  +  12(|s|  4-  If  |)2  +  (|s|  +  If  |)1  dt 


<  J  M  1^4  (Is)3  +  |f|3)  ■+■  48(s2  +  f2)  +  1st  +  |f|]  dt, 

by  inequality  (E2)  of  Appendix  E.  From  standard  results  for  moments  of  normal  distributions,  this  last 
bound  is 


|4s|3  +  3|4s|2  +  y|4s|  +  48  + 


Since  |4s  |  <  4Af  <  h  1/4,  using  this  bound  in  Eq.  (54)  we  can  show  that  the  second  term  in  Eq.  (49) 
is  bounded  in  magnitude  by 


'T+'*' 


6 Clhv*  h2  - 


for  sufficiently  small  positive  h.  With  inequality  (50),  this  also  means  that 

-4  +  l'l  ,  -T 

|£|  <  (2  +  6 hv*){lh2  -  y—-  <  4  O  A2  -T5-  eM  (55) 

V2w  2  V2tr 

for  all  h  less  than  some  positive  value. 

Lemma  3:  In  the  context  of  Eqs.  (21)  to  (23),  there  exists  an  h*  >  0  which  depends  only  on  the 
parameters  a  and  b  of  the  unperturbed  problem,  such  that  if  h  <  h*  and  if  |0|  < 

M  -  Wln(l/A),  then  p,  (0)  exists  and 


where 


|5(0)|  <  [3(1  +  Ml/F*  +  6, 000, 000)1  ew. 


5(0)  —  yfim  e  2  Ip,  (0)  —  p,  (0)1 


Pi  (0)  -  -  [1  +  y’lf+i  (0J  -  30)1. 

Proof:  From  the  definitions  of  c,  <r,  and  <b, 

Vm  -  «3tj,  +  3 +t2<r2. 
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Since  |x|  ^  In  (l/h)  and  m  —  — -.y1  >  a  by  assumption, 

J  i 


M  -  6j\n(\/h)  -*►  \$\  <  M  *+>  |0 1  <  -7  A~l/4 
Mi+ 1  4 


|0l  < 


1*1  < 


2hJa 


for  sufficiently  small  A  >  0 


I  4-  0jc 


i i/Vm 


1 

¥T 


The  conclusion  of  the  lemma  then  follows  from  the  results  of  Eqs.  (27),  (33),  (42),  (48),  and  (55)  and 
the  fact  that 

A  (0)  -  A  (0.  L)  +  p,(0.  I), 


because  the  condition  of  sufficiently  small  positive  A  was  invoked  in  the  derivation  of  these  results  only 
a  finite  number  of  times  and  in  such  a  way  each  time  that  'sufficiently  small"  depended  only  on  the 
values  of  a  and  b  in  the  corresponding  unperturbed  problem.  Thus  A*  here  can  be  chosen  as  either  the 
smallest  of  these  values  which  is  still  strictly  positive  or,  if  it  is  smaller,  the  maximum  h  for  which 


±ITV* 


1 


*  2  hJa‘ 


which  is  also  positive.  □ 


Lemma  4:  In  the  context  of  Eqs.  (21)  to  (23),  there  exists  an  A*  >  0  which  depends  only  on 
parameters  a  and  b,  such  that  for  every  h  <  A*, 


1  2 


Pr(s  ^  0)  <  (A  +  l)e  20*+1  ifff^  -7 A" 1/1  and  k  <  A~l/I2,  where  k  ■-  2/. 

4 


Proof-.  Let 


and  let  A  >  0  be  such  that  A  <  A* 

i  L-l 


—  M  -  6>/ln(l/A) 

Mi+i 

0  <  T7T  and  .0  >  j  A-,/,(see  the  proof  of  Lemma  3).  If 


¥T 


— A~1/8  <  0  <  0,  the  conclusion  of  the  lemma  follows  by  subtraction  from  the  integral  of  p,  and  a  of 
Lemma  3  for  sufficiently  small  positive  A  (the  maximum  allowable  value  depending  only  on  parameters 
a  and  A),  since 
the  two  events 


<  1.  For  0  >  0  >  ~A'I,,\  the  event  (0:  Is |  >  0)  is  contained  in  the  union  of 


E\  —  {t:  |r|  >  0  *+1 } 


and 


El-  kg--  1*1  >  0-0*+l  and  |rlO*+l}, 


W.W.  WILLMAN 


since  s  —  /  +  g  by  construction.  Since  m  >  v  >  0, 


M  >  fl*+1 


x  -  X 

~7T 


X  -  X 

~7T 


i*+i 


Hence, 


_±#  *+i 

Pr{£,|  <  2 


by  Assumption  2  of  Section  4.1.  Also, 

Pr(£2l  -  J 

but 


Pr||*i  >  0-0k+'/t)  dP(t). 


Prllsl  >0-0k*/t)-  20 


<  20 


<  20 


e  -  ek+1 

cr|l  +  4>t\ 

_l_  ' 

« *+1  -  l 


(O  -  normal  tail  as  before) 


1 


if  |/|  <  0  *+l ,  since  h  is  small  enough  that  0  >  -~  h 


i/» 


4h  12  +  h  j 
l7*7)  (if  0  >  (1  -  4A,/I2  -  A)-*-1). 


This  last  condition  is  satisfied  for  k  <  h  1/12  for  sufficiently  small  positive  h  because 
0  >  0  >  -j-  h~x,t  (for  sufficiently  small  positive  h) 


S<*  -t-  DA  12  >  e-(*  +  l)ln(l-5*,J) 


>  eb  >  e 


-  ei"Ki-s*'2)-*+,,l  >  (i  -  Sh  l2)_(*+l>  >  (1  -  Ah  12  -  A)-(*+,) 


for  sufficiently  small  positive  h.  Hence, 


Pr{|*l  >  9-9k+l/t)  <  -jLr 


for  h  small  enough  that  0^1.  For  0^0,  therefore, 


_±  g*  +  ! 
.  2 


Pr{£2)  <  j  e~2""  f 


UK*1 


under  these  conditions,  and 


0k  +  l  <  ~  e  2 


dP(i)  <  e  2 


V»‘+1 


PrJIsI  ^  0)<  Pr{ £ , }  +  Pr{£2)  <  (k  +  l)  e  2 

The  lemma  follows  because  h*  >  0  can  be  chosen  on  the  minimum  of  the  positive  values  h  and  the 
three  positive  maximum  values  of  h  needed  for  the  inequalities  used  in  this  proof.D 
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NRL  REPORT  8406 


From  the  definitions  of  <f>,  c,  and  a  in  terms  of  v  and  x,  from  the  Lipschitz  conditions  of  Lemma 
1  for  v,  x,  and  X,  and  from  the  results  of  Appendix  C  it  follows  from  partial  differentiation  of  Rt, 
rewritten  using  Eqs.  (14)  and  (30),  as 

/?,  -  lo-3«2(s6  -  12s4  +  21s2  -  6)  +  3(1  -  3<r2)s2], 

with  respect  to  c,  a,  <f>,  X,  and  s  (using  the  magnitude  bounds  for  and  v  assumed  in  Section  4.1)  that 
|*,(s,p)-  R\(s',p)\  <  I  ^2m»xl|i|. Ij’II  Vln(l//»)  h2  |s-s'|  (56) 

and 

|*,(s,  p)  -  R\(s,  p')l  <  j  e2UI  Ml/h)  h2\p  -  p'|  (57) 

for  sufficiently  small  positive  h,  where  the  arguments  of  R\  are  given  in  the  context  of  Assumption  4  of 
Lemma  2,  because  e|jrl  dominates  any  polynomial  in  x  for  large  enough  x. 


Lipschitz  Condition  for  I  j 
From  Eq.  (35), 


rL  e  2,7 

5  —  u 

4 

_  c 

s  -  u 

2 

+  2 

<r(l  +  ibfi) 

—  j 

<t(\  +M) 

u2  du 


+  JL  fL 
3^  J-l 


u‘ 

2«J 


V^2 


(«5  -  3«2w3) 


s  -  u 


<r(l  +*£) 


-  5 


s  -  u 


ff(l  + 


+  2}  du. 


(58) 


where  0 (u,  s)  €  10,  «]  U  [u,  0).  It  is  clear  from  the  computations  for  Eq.  (47)  that  the  second 
integral  in  Eq.  (58)  is  bounded  in  magnitude  by 

, _ _  -£  +  1*1 

Vln(l/6)  he  2 

for  sufficiently  small  positive  h  if  |s|  <  L.  Completing  the  square  in  the  exponent  gives  the  first 
integral  in  Eq.  (58)  as 


_*i 

e  2 

CL  e 

l+<r  J#(2+4) 

e  * 

u2 

s  -  u 

4 

-  5 

S-M 

H 

yThr 

'~L  Vl  +ct29( 2  +  9) 

y/lirv 

cr  (1  +  9) 

tr(l+0) 

(59) 


where 


e2s 


1  +a20(l  +  e)  ’ 


«V(1  +9 )2 


and 


1  +  <t29  (2  +  9)' 

9(s,  u,  p)  —  <f>(p)  0(s,  u)  (so  \o\  <  yffi  in  the  context  of  interest  here). 
If  the  variable  of  integration  in  Eq.  (59)  is  changed  to 


x  — 


u  -  u 
■Jv 
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then 


«2s 


«r(l  +  <t>  u)x 


1  4-  o-2^/* (2  +  <£/*)  >/ 1  +  2  +  ^/*) 


and 


(60) 


$  -  u  _ Zl _ ex  (61) 

cr(l  +  <t>p)  (1  4-  0/*)  U  +  <r2/*(2  4-  <£/x)l  V 1  +  tr2<f>p( 2  +  4>p) 

with 

/*(s,  x)  =/3ls,  u(x)]  (so0  =  d>/*).  ^2) 

The  partial  derivative  of  this  integral  (denoted  now  as  7)  with  respect  to  <f>  can  be  expressed  by 
Leibnitz's  rule.  The  resulting  integral  can  then  be  bounded  (by  standard  results  for  normal  moments) 
as 

1000  x  (polynomial  in  s,  <t>). 


since  s2  <  -~r  /Tl/2,  which  implies  that  |eSs|  <  2.  Then  by  the  definitions  of  the  variables  in  terms  of 
16 

x  and  v,  by  the  Lipschitz  conditions  established  in  Lemma  1  for  x  and  v,  and  by  the  results  of  Appendix 
C,  it  follows  by  routine  manipulation  that 

I  Tip)  -  Tip')  |  <  |>/iKn7Aj  e|j|  ip  -  pi. 

with  p  and  p  used  in  the  content  of  Section  4.1,  for  sufficiently  small  positive  h,  since  eM  dominates 
any  polynomial  in  s  for  large  enough  s. 


For  the  case  of  fixed  p  and  varying  s,  denote  the  integrand  of  T  by  H.  The  triangle  in¬ 
equality  for  integrals  gives  (for  si  <  s2,  and  Ujl,  ls2l  <  L  -  Wln(l/ h)) 


1 7W-  F(s2)|  <//* 


max 

U| 


41*  |  l,‘<S|) "  +  IH»r| |S| "  s,l|  * 

[cH+[c»‘ 


(63) 


where  [Lx,  L2\  is  the  common  x  range  of  integration  for  5,  and  s2,  and  where  H  is  regarded  as  a  func¬ 
tion  of  both  s  and/*  as  determined  by  substituting  Eqs.  (59),  (60),  (62),  and  the  change  of  integration 
variable  to  x  in  Eq.  (59).  Applying  the  sort  of  computations  involving  Eq.  (47)  to  Eq.  (59)  we  can 
show  that  for  |s,|,  |s2|  <  L  -  Wln(l//i)  the  last  two  integrals  in  Eq.  (63)  are  bounded  in  magnitude 
by  A3  for  sufficiently  small  positive  h.  Also, 

|/*(si,  x)  -pis2,x) I  ^  4(|x|  +  maxflsil,  |s2|)) 

by  the  triangle  inequality,  the  established  bounds  on  e,  o-,  I# I,  and  the  fact  that  1/3  d,  m)|  <  l«|. 
Using  these  inequalities,  standard  results  for  the  moments  of  normal  densities,  and  the  fact  that  I  s  I  ^ 

-7  h~l/*  and  |<£j  <  -ryTbh  allows  the  expression  of  inequality  (63)  to  be  bounded,  after  some  manipu- 
4  3 

lation,  by 

lVE(T7ATe2miX,l,,,-M  (A  +  lsi-s2l) 

4 
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for  sufficiently  small  positive  A.  Since  the  indexing  of  s,  and  $2  is  arbitrary  here,  this  also  holds  for 
s2  <  si-  Thus,  by  the  results  in  Appendix  C  for  Lipschitz  conditions  for  products  of  functions,  and  by 
the  Lipschitz  conditions  for  <j>  that  can  be  obtained  from  those  for  x  and  v, 


(s  i.p) 


±<t>2h 


(s2,  p ) 


<  \  jmTh)  eWlli|L  M  ( h 3  +  A 2  Is,  -  s2l)  (64) 


and 


■y*2/2|(si.p) -  |y<A2/: 


(s,  p') 


<  y-s/ln (l/ft)  e21’1  (A3  +  A2  |p  -  p'\) 


for  sufficiently  small  positive  A,  where  p  and  p'  are  as  in  Property  4  of  Lemma  2. 

Lipschitz  Condition  for  E 

From  Eq.  (36),  E  —  £,  +  £2,  where 


/L  u 

l°7- 


( t-u >* 

)  e'*2 

e  *2 

J  y/2 ire2 

•Jl  nar1 

dk„ 


and 


“(f)  75 


U1 

<J-u)2 

e'2*2 

e  2<TJ(l+4y)2 

s  —  u 

>/£? 

y/2tr<r2(l  +  d>y)2 

<r(l  +  «Ay) 

—  If  Udky. 


Completing  the  square  in  the  exponent  of  Eq.  (66)  gives 

.2  (l>-«*s)* 


-l 


e 
\/2n 


2  *L 

^  J-L 


>/2!ir<72€2 


dku. 


Denoting  the  integral  in  Eq.  (68)  by  £  gives,  for  fixed  (Zh  Y,- ,), 
|£(s,)-  £(s2)I- 


1 

f  ' 

<  1 

-1 

—  “  J  1 

«■  l 

«  *J 

X 

A-s} 

r  ° 

t  1 

J±i 

.L_. 

€  2 

V2  it 


a  (s 2  +  <rO)dkt 


(65) 


(66) 


(67) 


(68) 


,  (69) 


by  a  change  of  the  variable  of  integration  in  Eq.  (68)  to  $  : 


for  s  -  s,  and  s  —  s2 ■  Assum¬ 


ing  without  loss  of  generality  that  s,  <  s2  and  Is, |  <  |s2l,  the  right-hand  side  of  Eq.  (69)  is  bounded 
by 


/; 


i: 


l«(*i  +  <p9)  -  a(s2  +  <rO)\dkg  +  J*( 


1 

hs  ■ 

fit 

L-J 

'  2 


7*7 


|a  (s,  +  «rP)|</X# 
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1-1-*,  -y 

+ Jqi  1  ,,1  7ST  |a  (,i  +  <r,)  ,</x# 


for  |s,|,  |s2|  <  L,  because  0  <  c  <  1. 


For  the  case  of  interest  where  Is,  I,  |s2l  <  L  —  6>/ln(l/A),  the  last  two  integrals  in  this  bound  are 
clearly  less  than  h3  for  sufficiently  small  positive  h  because  of  the  magnitude  bounds  of  a.  By  the 
Lipschitz  conditions  assumed  for  a  in  Section  4.1,  the  first  integral  is  bounded  by 


-4-  *  2l»l 


(Ch3  +  Dh1  Is,  -  Sj|)  e2l*,+""  d9  <  (Ch3  +  Dh 1  |s,  -  s2l)  e21*'1  /“  -  dd 


<  2e*  ( Ch 3  +  Dh 3  Is,  -  s2|)  e21*'1  J#" 

<  2e*  (Ch3  +  Dh*\s\  -  s2|)  e21*'1. 


Combining  results  gives,  for  sufficiently  small  positive  h, 

|£(S,)  -  £(s2)|  <  [{le4C  +  l)hi  +  2e4m2  |ai  _  S2|] 

and,  by  applying  the  results  of  Appendix  C, 

|£,  (s,)  -  £,(s2)|  <  e2""11'1'- M((2e«c  +  1  )h3  +  he* Dh3  |s,  -  s2|], 
for  |s,  I  and  |s2|  <  L  -  6>/ln(l /h). 


(70) 


For  a  fixed  s  such  that  Is  I  <  L  —  &J\n(\/h)  and  p  and  p'  as  in  Assumption  4  of  Section  4.1, 
changing  the  variable  of  integration  in  Eq.  (68)  gives 

,  ,  li 

4.(1  -  *)  ~  2 


_  _  *-*)  2 
I  £(p)  -  £(pOl  -  L  ^  « (*  +  <rh,p)dkt 


cr'(9.y>0</A,| ,  (71) 

where  0  -  —  (—  -  s)  in  the  first  integral  and  -—(-4  -  s)  in  the  second,  and  o-'  is  the  value  of  <r 
<r  <  «r  < 

generated  by  (Z,.K(_ ,)'  etc.  Under  these  conditions,  the  regions  not  common  to  both  integration 
ranges  again  contribute  a  term  smaller  in  magnitude  than  h3  for  sufficiently  small  positive  h.  The  rest 
is  bounded  by 

_gl 

X,  <  o  + <r#,p*  ~ o(* + 

_#! 

+  Z)AJ(  |p-p'|  +  \«\  |cr- a'|)]eJ,*'e2,*l<W, 
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by  the  triangle  inequality  and  the  Lipschitz  conditions  assumed  for  a .  From  the  Lipschitz  conditions 
established  in  Lemma  1  for  x  and  v,  the  results  of  Appendix  C  can  be  used  to  show  that  \<r  —  <r'|  is  of 
order  A  Ip  -  p'|.  This  last  integral  can  then  be  bounded  as  in  the  derivation  of  Eq.  (70)  by 

4 e4(Ch3  +  Dh2\P  -  p'|)eJ|l) 

for  sufficiently  small  positive  A,  so 

I E(p)  -  £<p')l  <  [(4 e*C  +  1)A3  +  4*W  I  p  -  p'lje21*' 
and 

|£,(s,p)  -  £|(s,p')l  <  \&e*C  +  1)A3  +  Se4Dh2  I  p  -  p'l]*21'1.  (72) 

Since  I  Oy|~|  I  <  Va  for  sufficiently  small  positive  A,  it  follows  from  Eq.  (67)  that 

^ff  J‘“  -1 . 


+1  r1"'- 


Since  0  <  o,  c  <  1,  completing  the  square  in  the  exponent  gives  the  bound 

w  «k 


where 


TT^Va 


o'2!!  +  •Jli ) 


I  +  oVA 


Letting  —  andP-  —  and  changing  the  variable  of  integration  in  inequality  (74)  gives 

j2  |2  r—  {0,  0)2 

l£il  <  vk  /A  'V5^rl,<*)([ w7^n|  +1),l<"‘"'1 

Since  e  <  1  and  since  |a(0)|  <  flh2e^  in  this  range  for  |s|  <  L, 


.2  ^  _ ! 


\(r/h 


_  _j}  (t-i)2 


Changing  the  variable  of  integration  in  Eq.  (75)  to 
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we  get 


|£2I  <  kin 


1  +  yfh 

rll 

CXpi  2  *  *  |  €  5  •  (J 

1  +  <tVa 

1 

V2ir 


X  1  + 


er,+‘lrF6r  r 


ks  +  <T 


1  +  yfh 


1  +  <rVA 


tWt. 


(76) 


By  use  of  the  fact  that  cr2  +  e2  -  1,  the  magnitude  of  the  polynomial  factor  in  the  integral  of  Eq.  (76) 
can  be  bounded  by 

yUP  +  2s2|r|  +  yUU2  +  Is  I  +  2|t| 

and  the  nonquadratic  exponential  term  by  Ul  +  2|r|  for  sufficiently  small  positive  A ,  so  the  integral  is 
bounded  by 

-|(r  -  l)J 

f  — — j== —  (|s |3  +  4s2t  +  5|j|t2  -I-  2|s|  +  4T)dr 


•Jlit 


<  e 


I,  I 


|s|3  +  7|s|  +  4(s2  + 


”^1 


<  e^flsl  +  2)3 


by  standard  results  for  normal  moments.  In  this  case,  another  bound  for  this  integral  is  3e2|>l,  by  ine¬ 
quality  (El)  of  Appendix  E.  Hence,  for  |sl<£  and  sufficiently  small  positive  A, 

-4+n*i 


|£2I  <  4nk|/r2-~=- 


and 


|£2I  <  6n  A3 


viii 


,21,1 


(77) 


Lemma  5:  In  the  context  of  Eqs.  (21)  to  (23),  there  exists  an  A*  >  0  depending  only  on  the 
parameters  a  and  b  of  the  unperturbed  problem,  such  that  if  A  <  A*, 

|a(0,p)  -  a(0,p')|  <  e1|#,(CAJ  +  DA2  |p  -  p’|) 

and 

|5(b  .p)  -  a(0  ,p)|  <  g2imx(l*l,  1*11  (C/,3  +  Dh2\9  -  «'|) 

for  all  9  and  9"  with  magnitude  <  A f  -  6^^1n|-jjj,  where  a  is  as  in  Lemma  3  and  p  and 

p'  are  as  in  Assumption  4  of  Section  4.1,  and  where 

C  -  6fl  +  4  +  4e4C  +  3/4>/ln(l/A) 

and 

D  -  -|vin(l/A)  +  L  +  5 e4D. 
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Proof :  From  Eq.  (27),  \ffir  e  1  pis,  L)  <  y  h3  e2l,(  under  these  conditions.  Since  pis)  - 

pis,  L)  +  pis,  I),  the  lemma  follows  from  the  definition  of  a  in  Lemma  3  and  Eq.  (33)  and  the 
application  of  the  triangle  inequality  to  the  results  of  Eqs.  (41),  (56),  (57),  (64),  (65),  (70),  (72),  and 
(77),  because  the  condition  of  sufficiently  small  positive  h  was  only  used  a  finite  number  of  times  in 
these  derivations,  each  time  in  a  way  that  depended  only  on  the  parameters  a  and  b.  □ 


Because  of  the  way  the  random  variable  s  is  defined  in  terms  of  the  random  variable  y  of  Eq. 
(21),  Lemmas  3  to  5  establish  the  equivalent  of  Assumptions  1,  2,  and  4  of  Section  4.1  for  y,  with  the 


counterparts  of  x,  v, 


k,  C,  D,  and  M  being  Jr,  — 

Ji 


k  +  1,  C,  D  (as  defined  in  Lemma  5),  and 


4.4  Measurement  Update  at  Epoch  1 4-  1 


In  this  subsection,  the  unsubscripted  variable  x  refers  to  x,+i  instead  of  x(,  tj  refers  to  and 
the  subscripts  are  also  deleted  for  m(+1,  ri+j,  x,+,,  jq+(,  v,+1,  and  n(+1.  It  is  assumed  that  Assumptions 
1,  2,  and  3  of  Section  4.1  hold  at  epoch  /  +  1  conditioned  only  on  iZ„  yHl  •  u),  with  Jc,  v,  k,  and 
iZ,.  Y,-\)  replaced  by  x*  m,  k  +  1,  and  iZ,.  Yt-\  •  u)  in  the  notation  everywhere.  In  this  context 
z  —  x  +  so  defining 


g  •  cr  r,  (82) 

and 


s  -  t  +  g 
implies  that 

<r2  -f  « 2  —  1, 

-f1 


and 


Pi r)- 


V2ir 
z  -  x* 

Jm  +  r  ’ 

(j-Oi 


pis/ 1)  - 


e 


2 <r* 


vw1 


e 


(83) 

(84) 

(85) 

(86) 

(87) 

(88) 
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.lie  last  for  |r|  <  tM  in  the  current  notation.  It  follows  from  Eq.  (86)  that  conditioning  on  s  (in  addt 
tion  to  Zt)  is  equivalent  to  conditioning  on  z.  Hence,  if 

(89) 

for  any  choice  of  parameters  7 and  v,  then 

...  ,  yfv  pa/,is.  7  +  9-Jv)  p,G  +  9\fv) 

P(e/S) "  pis) 

(90) 

by  the  Bayes  rule  whenever  these  densities  are  defined.  If,  for  a  given  value  of  s  (or  z),  the  parameter 
values 

7  —  «2(s  +  <r2t  ij is2  -  1)1 

(91) 

and 

v  —  <r2«2(l  +  <r2tr\s)2 

(92) 

are  selected,  then 

t  +  9-Jv  —  tits  +  trot). 

(93) 

where 

to  —  tits  +  tr9). 

(94) 

Abo,  the  results  of  Section  4.3  can  be  specialized  to  the  case  of  Eqs.  (78),  (82),  (83),  (85),  and  (88)  to 
give 


~T  f 

pis)  -  Jl  +  1J€3(s3  ~  3s)  +  pis) 

for  |s |  <  *M  -  &J\ni\/h),  for  sufficiently  small  positive  h,  with 

1/3 1  <  3(lh2eM  (since  I2  -  0  now) 

in  the  context  of  the  current  notation.  Substituting  Eqs.  (88),  (92),  (93),  and  (95)  in  Eq.  (90),  we 
obtain 


(95) 


(96) 


pi9/s)  - 


e  2 

|l  +  ^-|(tJ  +  <rw)3  —  3(«s  +  trto)]  +  aits  + 

crw) 

(1  +  <r2etjs) 

y/2n 

1  +  jit«3(s3  -  3s)  +  Pis) 

(97) 


if  |s|  <  tM  -  6Vln(l//i)  and  |/|  -  |/  +  V7  0|  <  tM.  Since  t  and  v  are  specified  from  s  by  Eqs.  (91) 
and  (92)  now,  it  follows  from  the  triangle  inequality  and  limits  assumed  for  |t>|  that  these  two  condi¬ 
tions  are  met  if 

Is |  <  Viln(l//i)  (98) 

and 

l«  I  M  -  3Vln(l//r)  (99) 

for  sufficiently  small  positive  h.  From  the  definitions  of  s  and  Gh  it  is  clear  that  condition  (98)  is 
already  implicit  in  the  assumptions  here. 

Inituitively,  it  is  helpful  to  note  at  this  point  that  s  is  basically  the  normalized  "innovation"  vari¬ 
able  for  the  updating  step  of  a  Kalman-Bucy  filter  and  9  is  the  normalized  error  in  the  resulting  esti¬ 
mate  of  x. 
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Now  let  /  denote  the  function 


f(r\  1  +  gs£  ♦•••c**  ♦«<•*  -  iHl* 

Substituting  Eq.  (100)  in  Eq.  (97)  and  rearranging  terms,  we  get 

,  -U[(*s  +  <r«)3  —  3(«s  +  cr«)] 

p(9/s)  —  /(<r«ij)  - | - + - - - 

1  +  ytje3(s3-  3s)  +^(s)el11  1  +  yi)«3(s3-  3s)  +  0(s)ew 


+  —  a  (<s  +  <rw) 

V2w 


_ _ 

l  +  yt,(s3-3s)+/5(s)elfl 


for  conditions  (98)  and  (99)  and  sufficiently  small  positive  A,  with 

|a(fs  +  orci»)|  <  ft/r2 


|£(s)|  <  3ft  A2. 

Also,  / and  its  first  two  derivatives  are  continuous  for  all  (,  so  Taylor’s  theorem  with  remainder  can  be 
applied  to  /at  (  —  0  to  give 

/(<r«ij)  -  {1  -  <rCi)[<rs(02  -  1)  +  «(s2  -  1)0)}  +  — *  3--  /'Veip,  (102) 

V  €  I0,i jJ  U  f*|,  0J , 

by  the  use  of  <rrrj  to  play  the  role  of  {  and  substitution  from  Eq.  (94)  for  w.  Using  Eq.  (102)  in  Eq. 
(101)  and  substituting  the  expression  for  /*',  we  get 

p(0/s)- !  ^  (03- 30)  +  {1 -<r«»,(<rs(02- 1) +« (s2- 1)0)} 


*  |l  —  -  j-[e3(s3— 3s)  —  («s  +o,w)3+3(es-<ro»)J+£’|  +  £2-/®^)^*,| 

■y  +  k»+o,»l 

x  {1(0  +  <T€^)2  -  1)0  +  tr2tsri)y  -  2 <rs(9  +  o*^))  +  (1  +  £,)£4- - t— - - ,  (103) 

y/2ir 


where 


y(9)  -  <rs9  +  «(s2-  1) 
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|#  +  y<Tfrj  I  <  |fl|  +  -Jh  <  |0  I  +  1. 

(1  +  <r2«sij)  <  1  +  6hJ bln(\/ h)  <  2. 

|1  +  cr2tsrj  i  <  1  +  6 bJb  ln(l//i)  <  2, 

Ti  <  ih2b  (Isl  +  |0|)5  <  h2Ml«l  +  2V21n(l/h)l6. 


and 

T2  <  45/i2  6 (Is  I  +  |0|)4  Is  110 1  <  45/i2h  [|0|  +  2V21n(l//r)l6. 


Also,  the  last  term  in  Eq.  (103), 


(1  +  £j)  E4 


is  bounded  in  magnitude  by  2(1  h2e  2  e1*1  e^121"^**  for  sufficiently  small  positive  h  under  these  con¬ 
ditions.  But 


1/4 


—  — +|#| 

so  another  bound  is  e'/l3ln(1/',)  (ih2  e  1  .  These  inequalities  and  the  fact  that  polynomials  are  ulti¬ 

mately  bounded  by  exponentials  lead  to  the  following  conclusion,  after  some  manipulation. 


Lemma  6  :  In  the  context  of  the  notation  of  this  subsection  and  the  assumptions  described  at  its 
beginning,  there  exists  an  hm  >  0  depending  only  on  the  parameters  a  and  b  of  the  unperturbed  prob¬ 
lem,  such  that  if  A  <  h\ 

|^i|<  M-  Wln(l/h), 

I  Vp  1 


and 


<  ln(l/h), 


then  px_t  (9,  z)  exists  and 

^r/z-+' 


a*|  - 

*■ z)  -  737 

1  +  ‘3f“  (93  ~  3®) 

-£+w 

<  eVl4ln(l /h)  (ft  +  I)  tfL 


This  form  of  the  lemma  follows  by  the  construction  of  9  in  Eq.  (89)  and  from  the  fact  that  the 
choices  of  7  and  v  as  in  Eqs.  (91)  and  (92)  make  the  variables  here  correspond  to  those  defined  by 
Equation  System  I. 

Lemma  7:  In  the  context  of  the  notation  and  assumptions  described  at  the  beginning  of  this  sub¬ 
section,  if  it  is  also  true  that  Assumption  4  of  Section  4.1  holds  for  (  and  any  two  values  of  a  single 
component  of  (Z„  K,_i  •  u,)  and  (Z„  Yt. i  •  «,)',  then  there  exists  an  h*  >  0  such  that  if  h  <  h*. 
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|-*^|<  M-3MI/H), 

and 

1  vm  +  r  l 

then 

|a*(0,  £)  -  a*(6',  01  <  (Oi3  +  £>h2\B  -  0’|) 

for  Itf'l  <  M  -  3Vln(l//t) 

and 

I <*•(».  0  -  a*(9.  £')|  <  em  ( Ch 3  +  £/t2|£  -  £'|), 
where  a  *  is  as  defined  in  Lemma  6  and 

C  -  eVuin(1/*)  (C  +  1) 

and 

D  -  e'/i4ln(Vl,)  (D  +  1). 

Proof  :  The  results  of  Section  4.3  can  be  specialized  to  the  context  of  Eq.  (95)  to  give 
1/8  (s,  0  -  pis,  £')l  <  22|l!  (Ch3  +  Dh2\(  -  £'l) 

and 

Ipis,.  ()~P(S2.  C)l  <  e2n*K,l*i|*  M  (C/,3  +  3h2ls,  -  S2I) 
for 

C  -  4e4C  +  1  +  y  Vln(l//») 
and 

D-5e*D  +  1  +  yVln(lM). 

From  the  Lipschitz  conditions  already  assumed  for  a  and  the  fact  that 

a(£)  —  a(f)  e~^ 
and 

p(()-p(t)  e-l{l, 

it  follows  from  the  results  of  Appendix  C  that 

l«(f.  0  -  o(f ,  £')l  <  e*fl  (Ch3  +  Dh2\l  -  £'|) 

la(f.  0  - &((’.  ()|  <  W  [Ch3  +  (D  +  fl)  h2  If  -  f'll 

I  Pit.  0  -  Pit,  {')l  <  elfl  (Ch3  +  Dh2 1£  -  £'i) 

\p((,  0  -  Pit’.  01  <  en“*0{,-l<,,)  la3  +  (D  +  30)  h2\(  -  OU 

for  the  conditions  of  interest  here.  The  lemma  is  then  established  by  repeated  application  of  the  results 
of  Appendix  C  to  the  composite  expression  of  Eq.  (103)  for  p(9/s),  using  the  definition  of  0  and  s  in 
terms  of  the  variables  of  interest,  various  inequalities  developed  earlier  in  this  subsection,  and  the  fact 
that 
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and 

max 
{>0 

To  establish  probability  bounds  for  extreme  values  of  the  estimate  error,  suppose  first  that  y  is  a 
value  such  that 

y  >  z  -  x*  >  0, 

and  for  small  A  >  0  consider 

Pr{x  -  x*  >  y/x  +  n  €  [z,  z  +  A)}. 

From  the  definition  of  conditional  probability,  this  probability  is  bounded  by 

_ Pr(x  -  x*  >  y  and  x  +  n  €  lz,  z  +  A)} _ 

Pr{x  +  n  €  lz,  z  +A)  and  2z  +  A  -  y  <  x  -  x*  <  y)  ‘ 

Since  x  and  n  are  independent  (given  only  2,  and  Y,),  and  since  n  is  normal  (0,  r),  the  numerator  of 
this  last  expression  is  bounded  above  by 

(x'+v-z-*)1 

A  — - 7=== -  C  dPix) 

V2 nr  J*'+y 


+  <ro» )  -  its  +a-0)|  <  hy/lni\/h), 

4-  Us  +  oo) (0)1-0  x  0(h), 
dt 


{{'*-<}-  |f 


for  p  ^  1. 


and  the  denominator  is  bounded  below  by 

( x'+y-z )2 

p  if  r*'+y 
4  - dPix) 


Hence, 

Pr{x-x*  >  y  /  (x  +  n)  €  [z.  z  +  A]}  < 
Taking  the  limit  as  A  — *  0  gives 

Pr(x-x*>  y/z)  < 


_AUz£±i!l 
e  r  Pr  (x  —  x*  >  yl 


1  —  Pr{x  —  x*  >y}-Pr{x~x*  <  2(z-x*)-y  +  A) 


Pr(x-x*  >  y) 


l-Pr(x-x*  >y}-Pr{x-x*  <  2(z  — x*)-y} 

For  Pr{x-x*  < -y/z),  this  construction  shows  that  the  inequality  is  at  least  as  great  as  (104).  Simi¬ 
larly,  for  y  <  z  -  x*  <  0, 

Prlx  “  x*  <  y/z]  *  1  -  Pr{x  -  x*  >  y)-  Pr{x  -  x*  >  2(z  -  x*)  -  y) 

and  there  is  at  least  as  great  an  inequality  for  Pr{x  -  x*  >  -y/z).  Combining  these  results  for 
y  >  3|z-x*|  gives 


PrlU-^|  >yh)  <  _.NI*r»*l>/l 

1  —  Prl  l^c  —  jr*  I  >  y  jj 

where  all  probabilities  are  also  conditioned  on  Z,  (not  shown  in  the  notation). 


(105) 
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From  Equation  System  1, 

x  -  x*  —  0c  —  Jr)  +  (Jr  —  x*)  —  9^/v  +  ■  ^  -  + 

yjm+r 


rrim 


3/2 


(m+r)2 


(sJ—  1), 


where,  as  before  in  this  subsection, 


and 


$  - 


x-x 

~7T' 


For  y  >  0,  therefore, 

\9\>  y  |x-x*|  >  yJv  -Vm  |s+ij(s2-  t)| 

-►lx— x*|  ^yJa  —  yjb\n(\l h) , 

for  sufficiently  small  positive  h ,  because  of  the  bounds  assumed  for  |m|,  |i)|,  and  |s|.  Hence,  for 
sufficiently  small  positive  h  andy  >  In  (1/A), 

|0|  >  y  |x  —  x*|  ^  ~  yja.  (106) 


By  assumption,  |z  -  x*|  <  V8A  ln(l/A),  so  for  sufficiently  small  positive  A, 


>  4- A  *  ■►  >  3|z  -  x*|. 


8 


In  this  case,  inequality  (105)  can  be  applied  to  (106)  to  give 


Prllfll  >y/z)  < 


*  +  1 


1  —  (A  + 1)  e 


_iUkr+l 


2  64 


*+l 


2|  24  J 


(107) 


by  use  of  the  inequality  established  in  Lemma  4.  Furthermore,  for  sufficiently  small  positive  A, 


/<y>/ln(l/A) 


I  *-•/* 


y>jh 


^lny>—  ln8+-|-  ln(l/A)  >  (2/  +  2)  In 


2b 

a 


*,>  a 


2/+2 


2  2<+2 

-►  >  O'1) 2,+J  (since  k  -  2i  +  1) 

2D 


-ilj'i*)**1’  _I, 

-►e2*2*1 


Jk+2 
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and 


i<  -p/ln(l/A> 

,>}r« 


Iny  >  -  ln8+  -t  ln(l/A)  >  Injs-^-J  +  (2i  +  2)[ln4+lnln(2i+2)l 
[4ln(2/+2)]J'+2 

f2'6*'  >(*  +  !)(* +  2) 


-*•!-(*  +  1) 


*  +  i 


k  +  1 


1 

k  +  2 


k  +  1 


_I  Uk1 

66 


<  k  +  2. 


*+i 


1-  (*  +  l)e  21 

Hence,  using  Eq.  (107)  and  the  definitions  of  8  and  s,  we  establish  the  following  lemma. 


Lemma  8:  In  the  context  of  the  notation  and  assumptions  adopted  at  the  beginning  of  this  subsec¬ 
tion,  there  exists  an  h*  >  0  depending  only  on  the  parameters  a  and  b  of  the  unperturbed  problem, 

such  that  if  h  <  h*  and  the  epoch  index  i  is  less  than  -yVln  (1/a),  then 


>  y/Zi+i 


<  (k  +  2)  e  2 


_±  „  *+2 


for  ally  > 


I 

8 


A-V». 


In  summary,  the  results  of  this  subsection  establish  counterparts  to  Assumptions  1,  2,  and  4  of 
Section  4.1  at  epoch  i  +  1  if  i  <  yVln(l/A). 


4.5  Implications  for  a  Sequence  of  Epochs 

If  an  error  is  multiplied  by  a  factor  not  exceeding  a  constant  H  in  magnitude  at  each  epoch,  and 
an  additional  error  not  exceeding  H  in  magnitude  is  added,  it  can  easily  be  shown  by  induction  on  N 
that  the  accumulated  error  after  N  epochs  is  less  than 

(H  +  0* 

if  the  initial  error  is  zero.  Hence,  if 

N  <  -j^J-ln(l/A),  0  a  constant, 
and 

H  «  e^Wn)  -  1, 
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then 


(H  +  l)N  ^ 

<  h~V2 

«  7  h~“‘  f"  *<  -k- 


Theorem  1:  Given  the  restrictions  imposed  on  the  unperturbed  problem  in  Section  3.1,  if  the 
number  of  epochs  N  is  such  that 

|  y/foll/h), 

and  if  (Z,,  Yt-\)  €  G,  (where  /  €  {0,  1,  ....  N)),  then  Assumptions  1,  2,  and  4  of  Section  4.1  hold 
for  epoch  /  in  the  problem  with  perturbations,  if  h  is  smaller  than  some  strictly  positive  value  h*  which 
depends  only  on  the  parameters  a,  b ,  and  F of  the  unperturbed  problem  (and  not  on  the  epoch  index). 


Proof  (induction  on  i) :  These  assumptions  are  clearly  true  at  epoch  0,  with 
Af0  —  -j  h~x)*.  fl0—  Co  -  Z>o”  0,  and  h*  -  2~32.  Assumption  2  follows  for  h  <  2~32,  by  the  in¬ 
equality  for  normal  tails. 


_£  _£ 
f "  e  2  ,  e  2 

J, 


forT>0- 

The  induction  step  can  be  established  by  noting  that  x/+1  can  be  constructed  from  x,  as  the  composition 
of  the  two  transformations 

and 


y  -  x,  +  (l  ++,xi)w 
*/+ J  -  f\y  +  U„ 


-IT*  <  I  ' 

where  w  -  y-,  a  normal  |o,  -^-J  random  variable.  Since  (Z,  •  z,+ j,  T,_|  •  «,)  € 
*■►  (Z„  T/_|)  €  Gh  if  it  is  assumed  that  Assumptions  1,  2,  and  4  of  Section  4.1  hold  with 

0/  ^  A), 

Q  ^  Ai, 

and 


J/+i 


A  ^  A/, 

it  follows  from  Lemmas  2  through  5,  applied  to  these  two  transformations  in  turn,  that  it  holds  at 
epoch  i  +  1  with  conditioning  only  on  Zh  for  corresponding  constants 

0  -  3A,  +  hj-^-  +  6,000, OOoJ, 

C  -  (6  +  4 e*)A,  +  Vln(l//r), 

D  -  5 e*A,  +  Vfnd/h), 
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and 

M  -  A/,-6%/ln(l/A) 

if  A  is  smaller  than  some  strictly  positive  quantity  that  does  not  depend  on  /,  since 

/  <  j  Vln(l/A>  -►  /  <  j 

for  sufficiently  small  positive  h.  Also,  the  parameter  k  -  2/  +  1  changes  to  2/  +  2  here.  If 

M  >  h  Lemmas  6  through  8  imply  that  the  hypothesis  also  holds  at  epoch  /  +  1,  with  condition¬ 
ing  on  Z,+ 1,  for 

A//+,  -  A?  -  3>/ln(l//i)  -  s/i  M,  -  9>/ln(l !h), 

n/+,  -  ev,4b,a /*>  (n  + 1). 

C(+,  -  ^Ulnd/A)  (£  +  1), 

and 

A+l  -  ^MWI/«  (5  +  1). 

Thus,  for  sufficiently  small  positive  h,  a  corresponding  bound  A,+\  is 

-  «wwl/*>  (A,  +  1). 

By  the  remarks  at  the  beginning  of  this  subsection,  therefore.  ft,,  C,,  and  Z>,  do  not  exceed  4-  /Tl/4, 

1  ^ 
for  sufficiently  small  positive  h  and  i  <  -j-  Vln(l//i),  and  it  is  also  clear  that  such  a  positive  upper  limit 

for  h  exists,  so  that  Af,  >  -j  h~ 1/8  for  A/0"*  -j  h~l/*  and  /'  <  -5-  y/ln(l/h).  Since  the  condition  of 

sufficiently  small  positive  h  was  used  in  this  proof  only  a  finite  number  of  times,  in  a  way  which  did  not 
depend  on  the  index  /  of  the  induction  step,  and  depended  only  on  the  parameters  a,  b ,  and  F  (some¬ 
times  via  Lemmas  3  through  8),  the  induction  step  is  verified.  □ 


4.6  Moment  Error  Bounds 

In  this  subsection  it  is  assumed  that  (ZN,  YN- ()  €  GN,  and  the  first  four  central  moments  of 


if  they  exist,  are  denoted  by  xh  <r},  28h  and  yt  respectively  for  epoch  /’  <  N.  For  a  given  epoch  /,  with 

i  <  |  Vln  (1  /h) 
and 

7  /r‘/8  <  M,  <  7  /r'/4, 

4  4 

we  suppress  the  epoch  subscript  and  the  conditioning  on  Z,  in  the  notation,  and  note  that  (Z,,  K,_i)  € 


i 
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Lemma  9:  In  the  current  context,  there  exists  an  h*  >  0  such  that  for  A  <  A*  and  n  -  1, 2, 3, 4 

XJ  <■  «w»>  <  **• 

vv 

Proof.  Since  Pr  |  |  >  /J  <  ke  *  ,  *  -  21  +  1  for  |r|  >  A/,  it  follows  from  the  construc¬ 

tion  of  the  Lebesgue  integral  and  the  monotonicity  of  r"  on  [A/,  <*>)  that 

/•»»  /•  “  — 

0<J„  tH  dPx_s(t)  <  k  jM  re  2  dr, 

v» 

where  it  -  2/  +  1  <  l/2Vln(l/A).  Changing  the  integration  variable  to  u  -  r17*  gives 

JJ  ^  (»)  <  *2 

~77 

Integration  by  parts  gives  this  last  bound  as 

.a.l-2  -LuVk  _«i 

it2  {A/  *e  2  +  t(«  +  1)A  -  2]  «("+,)*-3  e  2  dv). 

Hence,  it  is  easy  to  show  by  induction  on  k  that 

/at  ,n  (»)  <  +  2)*("+,)“2  e  2  . 

"TIT 

For  sufficiently  small  positive  h,  k  <  1/2  Vln(l/A),  M>  1/4A“V*,  and 


in  which  case 


s  si.  (*■/»>  T1"^-21"4 

A/y*  >  (4AVI)  '/ln(,/*)  -  e  V|"W*>  -  e  Vln  (I/*)  >  2, 

f  r"  dP,  ,  (r)  <  A2  2*("+,)  A/"+1  e  2  <  ife2et("+,)  A/"+1  e  2 

7T 


The  natural  logarithm  of  this  last  bound  is  2  In  it  +  *(n+l)  +  (n+1)  In  M-  -jA/^*,  which  is  less 
than 

2  In  l>/to  (1/A)]  +  (»+l)  iVln  (\/h)  +  -i-  In  (1/A)]  -  j  |iJw,n0/*) 

under  these  conditions.  For  sufficiently  small  positive  A,  this  expression  is  dominated  by  the  last  (nega¬ 
tive)  term,  which  can  be  rewritten  as 

_  -L  -  Win  (V*) 

8  e 

and  can  clearly  be  made  less  than  -  3  In  (1/A)  by  taking  A  to  be  a  small  enough  positive  value.  Since 
this  is  the  logarithm  of  the  bound  of  interest, 

>’  «>  <  *’ 


for  all  A  <  A*  for  some  A*  >  0.  □ 
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In  the  case  of  the  first  central  moment. 


£(x )  -  x  A  jf  +  yfv  E  1^7^ 


where  E  denotes  expectation  (conditioned  on  Z,)  and  x  and  v  are  as  generated  by  Equation  System  I. 
Thus, 

x  -  x  -  ^  J”  tdPx..x  it) 


■Jv  f  ^  ipit)dt  -  JA  ipit)dt  +  tpit)dk,  +  fA  tdPx_x  (r) 


(108) 


where 


pit)  -  -^-=-  [1  +  -3-(r3  -  3r)l  (the  approximation). 
A  -  (-  oo.  M]  U  [3/,  °°). 


and 


pit)  -  px_jf  it)  -  p  it)  (the  error  function). 

7 7 

Also,  there  are  no  moment  existence  problems  because  of  Lemma  9.  From  the  results  of  earlier  sub¬ 
sections, 

_£ 

1/8  (f)  I  < 


for  |r|  <  M and  sufficiently  small  positive  h.  The  first  integral  in  Eq.  (108)  is  zero,  so  from  this  ine¬ 
quality  and  that  established  for  hi  when  iZh  Y,-i)  e  Gt, 


\x  —  x  |  <  Vb 


2  f,  (r  +  hJb  it 4  +  3r2)]  dt 

V2w 


(109) 


The  first  integral  in  Eq.  (109)  can  be  made  less  than  h3  for  sufficiently  small  positive  h  by  standard 
results  for  normal  tails,  and  the  second  integral  is  ir-t/2.  By  Lemma  9,  therefore. 


Ix-Jcl  <  y/b 


hi1. 


(110) 


With  respect  to  the  conditional  distribution  given  Zh 

Eix  -  x)2  A  cr2  -  (x  -  jc)2  + 


(111) 
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and 


(/) 


J_I  JA  t*pU)dt+  /"  dt+JA  t2  dP (r). 

'JZ 


Using  Lemma  9  with  n  —  2  and  the  same  type  of  reasoning  as  for  the  first  central  moment,  we  can 
show  that  this  last  expression  is 

1  +  error,  where  lerrorl  <  (W3  0  +  3)  A2. 


For  sufficiently  small  positive  h,  (3c  -  x)2  is  of  order  A4,  and  so 

|<r2-  v|  <  ft(W5  0  +  3)  A2  (112) 


By  the  preceding  results, 

E(x  -  x)3  A  2#  -  3(x  -  3 c)2£(x  -  x)2  -  2(x  -  3c)2  +  v4'2 


-HM1 


Ml 


Repeating  the  above  analysis  using  t3  and  n  —  3  we  get 


29  -  2vV2tj  +  error,  lerrorl  <  bVi  16  -j-  +  3  ; 


where  ij  is  as  generated  by  Equation  System  I.  By  the  definition  of  X ,  therefore. 


I®  —  x  I  < 


80  ,  3) 

|7r+yJ 


t*1*1 


for  sufficiently  small  positive  h. 


(113) 


Again  expressing  x  -  3c  as  (x  -  x)  +  (i  -  3c), 

£(x  -  3c)4  A  y  *  £l(x  -  i)4  +  4(3c  -  3c)  (x  -  x)3  +  6(x  -  x)2  (x  -  3c)2] 

+  4(3c  -  3c)  (x  -  3c)3  +  (3c  -  5)4.  (114) 

From  the  preceding  results  for  lower  moments  and  the  fact  that  |rj|  <  3  bib  under  the  present  condi¬ 
tions,  the  only  term  in  Eq.  (114)  that  is  significant  to  order  A  is 

£(x-x)4-  vJ  /”  r4  dPx_s  (r) 

-  3v2  +  error,  lerrorl  <  b2h 


by  a  similar  analysis.  So 
for  sufficiently  small  positive  h. 


ly  -  3vl|  <  b2h 


(115) 


44 


Lipschia  Conditions 


NRL  REPORT  1406 


If  for  a  given  epoch  /,  /  <  N,  (Z„  Y,-iY  €  (7/  differs  from  the  initial  segment  (Z„Y,-\)  of 
(ZN,  Yn-i)  in  only  one  component,  for  which  the  respective  values  are  denoted  as  p'  and  p,  it  follows 
from  Eq.  (108)  and  the  analysis  in  this  subsection  that 

8j(p)  -  8i(p0  -  Vv(p)  [«i  +  J_M  tfi(t.  p)  A,\  -  Vv(pO  [*2  +  tft(t,  p*)  <fX,J,  (116) 

where  i\  -  3c  -  x  and  v(p 0  denotes  vl(Z„J',_i)'],  etc  with  kil  and  |«2|  <  3A3  for  sufficiently  small 
positive  A.  Rearranging  terms  in  Eq.  (116)  gives 

«i(p)  -  8i(p0  -  >/v(p)  jcc!  -  «2)  +  J_M  t\fi(t,p)  -  fiU.p'))  </X,J 

+  DfipO  -  x(p0]  h/v(p)  -  VvipOl.  (117) 


Since 


0O.p)“  <*Up) 


by  definition,  it  follows  from  the  Lipschitz  conditions  established  by  Lemma  1  and  Theorem  1  for  v  and 
a  (using  also  the  results  of  Appendix  C  in  the  case  of  v)  and  from  inequality  (110)  that 


-4+2, 


|#,<p)  “  8i(p0l  <  yfb  6A3  +  2 /"  t(Ch3  +  Dh2\p  -  p'|)  e-J—  dt 


♦IHI 


r 

2 7Z 


A*lp  -  pi 


2v 

(118) 

8)  then  gives,  for  sufficiently  small 


D  +  l|AJ|p-p!  (119) 

(120) 


for  sufficiently  small  positive  A.  Performing  the  integration  in  (11 
positive  A, 

l8i<p)-«i<p')l  <>/a[6+c2|2+^/J|cJ  Aj+[>/A*2[2+.n/XJ 

Since  C  and  D  are  less  than  A-V4, 

|8,(p)  -  8,^0 1  <  (2A"V4)  A3  +  (2/t~V4)  AJ|p  —  p'i 
for  sufficiently  small  positive  A. 

From  Eq.  (Ill)  and  the  fact  that  (x  -  x)2  <  -yA3  for  sufficiently  small  positive  A, 

*2<p)  “  *2 (p')  -  v(p)[«3  +  f_M  t20(t,p)  dX,]  -  v<p0[«4  +  £.*  t20(.t,p’)  Af]. 
with  |«j|,  |c4|  <  A3,  where  #2  -  <r2  -  v. 

Rearranging  terms,  using  the  triangle  inequality,  and  using  the  magnitude  bounds  and  Lipschitz  condi¬ 
tions  established  for  a,  we  obtain 


|«2(p)  -  «2(p')l  <  v(p) 


2A3  +  t2  | a(t,p)  -  a(t,p’)\dk, 
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|v(p)  -  v(p')| 


+ 1*1 + 5-u ,J  iff;  d*> 

<  A^A3  +  /“  HCh3  +  Dh2\p  -  p'|)  e  -  rf/j 

I  -  -t"1-1'1  I 

+  U  +  n  r2  -g-^~  dj  r*3|p  -  p'l 


for  sufficiently  small  positive  A.  Evaluating  the  integrals  in  this  last  inequality,  we  get 
l*2<p)  -  #2<p')l  <  *[2 A3  +  e2  (■••VH  (CA3  +  Dh2\p  -  p'l) 

+  r a3|a  +  0^(2  +  Ip  -  p'l 

in  this  case.  Also,  for  sufficiently  small  positive  A, 

l*2<P>-«2<p0l  <  (2A”V4)A3  +  (2h~v*)h2  lp-p'|, 
because  C  and  D  are  both  less  than  h~v*. 


(121) 


(122) 


Since,  as  before,  the  condition  of  sufficiently  small  positive  A  was  invoked  only  a  finite  number  of 
times  in  their  derivation,  Eqs.  (110),  (112),  (113),  (115),  (119),  and  (121)  can  be  summarized  in  the 
following  result. 

Lemma  10:  There  exists  an  A*  >  0  such  that  if  A  <  A*,  and  if  (Z„  f,_|)  and  (Zj,  T,_|)'  €  G, 
differ  in  at  most  one  component  with  respective  values  denoted  as  p  and  p'  for  /  <  N  <  -5-  VfciO/A), 

-  O 

then  the  first  four  central  moments,  x„  erf,  29,,  and  y,  of 


p*i  -  »« 

"7T 


exist  and 

l*i  -  $|  <  ft,+3j  A2, 

w?-v,\<  bMn,+3)h2, 

|d(  — A/I  <  Ayi|-^-fl/  +  3j  A2, 

|y,  -  3v,a|  <b2h, 


Ia,(p)-*,(p')l  <  >/F|6  +  e2|2  + 

and 


V?HaJ"  ^VXJa+^Ip-pI, 


l*2<P>  -  «2<p0l  <  *[2  +  e2[s  +  &^/J)c,]a3  +  *eaJs  +  +  l]  A2  |p-p'|. 
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where  xh  vh  and  A/  —  vi3/2Vi  are  as  generated  by  Equation  System  I  and 

*i  “  “  x, 


and 

*2  ”  <r }  ~  vt. 

S.  ADEQUACY  FOR  OPTIMAL  CONTROL  APPROXIMATION 

One  common  use  of  state-estimation  results  is  in  the  implementation  of  optimal  control  laws.  It 
is  shown  in  this  section  that  the  error  bounds  established  in  the  previous  section  are  strong  enough  to 
guarantee  that  the  control  values  computed  in  such  a  way  according  to  a  certain  first-order  formal 
analysis  (to  be  described  shortly)  are  indeed  within  order  h  of  being  the  optimal  values,  except  perhaps 
for  a  set  of  realizations  of  small  probability,  when  we  minimize  the  expected  value  of  a  quadratic  per¬ 
formance  criterion  with  the  dynamics  of  Eq.  (1)  —  u,  being  the  control  variable  —  and  with  the  state 
measurements  of  Eq.  (2).  Also,  the  set  of  realizations  for  which  this  control  accuracy  does  not  hold  is 
shown  to  be  contained  in  another  set  of  realizations  for  which  the  prior  probability  goes  to  zero  as  h 
does  and  it  is  always  known  to  the  controller  from  available  data  whenever  the  current  initial  segment 
of  a  realization  is  such  that  the  realization  cannot  be  in  this  set. 


5.1  Control  Problem  Formulation 


The  particular  control  problem  considered  here  is  that  of  minimizing  the  performance  criterion 


y- 


(123) 


where  E  denotes  prior  expected  value  and  the  x,  are  determined  by  Eq.  (1),  with  the  u,  now  regarded  as 
control  variables.  Also  sN  >  0  and,  for  /  -  0,  . . ,  N—  1,  A  >  a,  >0  and  b,  ^  B  >  0.  The  control 
variables  are  generated  from  the  available  data  by  a  Borel-measurable  control  law  U,  i.e., 


U:  (Zt,  F,_,)  — *  uh  <-  0 . N-  1. 


(124) 


Thus,  the  convention  used  here  is  that  the  controller  receives  the  measurement  z,  at  each  epoch  / 
before  having  to  decide  the  value  of  the  control  u,.  Also,  we  let  f  denote  the  set  of  admissible  control 
laws,  i.e.,  those  having  the  functional  dependence  indicated  by  expression  (124)  such  that  these  func¬ 
tions  are  Borel  measurable  (so  that  all  quantities  determined  thereby  are  well-defined  random  vari¬ 
ables). 


A  particular  control  law  U0  €  (  is  called  optimal  if  and  only  if 

J{UJ  “  inf  {/((/)  :  U  €  (). 

This  infimum  clearly  exists  because  J  is  necessarily  nonnegative  by  its  definition  and  J  is  obviously 
finite  under  the  trivial  (and  admissible)  control  law  which  always  gives  u,  -  0.  For  a  given  epoch  /  and 
data  value  (Z/f  Y,_i),  a  control  u,  is  called  optimal  if  and  only  if  there  exists  a  U  €  (  such  that  U  is 
optimal  and  u,  -  U(Zh  F,_i) . 


that 


Also,  it  is  convenient  to  define  the  sequence  of  optimal  value  /unctions  V„  i-  0 ,  N,  such 


V.iZ'.Y'-O-  inf 


Y  W  +  %  -J  (a,x/+&,u/)J 


(125) 
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when  (Z/.J^-i)  is  such  that  this  expectation  exists,  where  Uj  denotes  U(Zj,  Ky_i).  Heuristically,  V,  is 
the  usual  conditional  expected  cost-to-go  function  at  epoch  /,  given  the  currently  available  data  but 
before  the  control  «,  is  used. 


5.2  Equation  System  II  for  State  Estimation 


For  the  purpose  of  analyzing  this  control  problem,  it  is  more  convenient  to  adopt  a  somewhat 
different  system  of  equations  for  generating  approximate  moments  of  the  conditional  state  distribution, 
even  though  this  requires  some  effort  to  establish  some  relationships  between  the  two  equation  sys¬ 
tems.  This  other  system,  denoted  Equation  System  II,  is  the  following  for  /  -  0, ... ,  N—  1: 

4+i  “  [— — IJr~ — I  •'/♦i  +  */+i  ;  d0  —  0; 

lM<+i  +  r/+il  O+i 


(126) 


X,+l  -  fx,  +  u,  +  |u,+  1  + 


2r/+ 1  v/+1 

«/+ 1 

.  ^f+1 

M/+1  +  '(+1 

Mi+I  +  '/+! 

*0-  *o; 


(127) 


and 


^<+i  ” 


O+i 


M/+1  +  0+1 


</A/  +  3 fp,qth,)\  X0  -  0. 


(128) 


where  mi+i  and  p,  are  as  defined  by  Eqs.  (3)  and  (4),  and  where 

»0+i  -  f2d,  +  qri>,x, 

and 

O+i  **  0+1  *“  ftXt  —  U/. 


(129) 

(130) 


In  this  system,  the  p.,.  ph  and  X/  are  actually  just  predetermined  parameter  sequences;  only  the  d,  and 
%  are  random  variables  a  priori.  This  system  is  constructed  to  give  the  following  first-order  approxima¬ 
tions  to  the  variables  generated  by  Equation  System  I,  in  the  form  of  Eqs.  (7),  (8),  (10),  (11),  (16), 
and  (17): 


x,  x„ 
p,  +  2d,  =  v„ 
X/  as  X/, 


and 


Pi  +  2v,  =  m,. 


In  the  present  context,  of  course,  the  u,  values  are  not  generally  a  known  sequence  a  priori, 
because  the  control  law  is  allowed  to  generate  them  by  feedback  from  the  state  measurements.  Because 
of  the  nonanticipative  nature  of  this  feedback,  however,  all  previous  control  values  are  known  at  each 
epoch,  so  the  state  estimation  results  of  Section  4  can  clearly  be  applied  to  this  case  as  well  if  we  take 
N  -  i  at  each  epoch  /. 


5.3  Formally  First-Order  Optimal  Control  Law 


The  analysis  of  Ref.  1  can  be  applied  here  to  give  approximations  to  an  optimal  control  law  and 
the  value  functions  which  are  formally  accurate  to  first  order  in  h.  In  this  particular  case  the  approxi¬ 
mation  for  the  control  law  reduces  to 
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sl+lfl*l  +  */+ 1 


S/+j  +  b, 


;  i  -  0 . N-  1; 


where  &,  denotes  the  control  generated  by  this  law  at  epoch  /,  x,( Z„  Y,)  as  generated  by  Eqs.  (126)  to 
(130),  and  the  other  parameters  are  defined  by  the  recursions 


Yk  “  fk 


ak  + 


**+ 1 


fkbkSk+l 


bk  +  5*-h 


;  s/v  as  given; 


and 


**- 


&*  +  **+i 
/*****+! 


+  1- 


P*+l  |2 


+  **+l 


+  U*+t  +  |i  ~ 


r*+ 1  J 


Pk+l 

rk+ 1 


-  0; 


J't+il  “  0; 


with  X  —  N  -  l,  N  -  2 . 0  and  with  a+i  as  defined  earlier  by  Eqs.  (3)  and  (4). 

The  corresponding  approximation  to  the  optimal  value  functions  is  given  by 


1 


1 


*;_,)  -  —  s,(x,  +  Pi  +  2d,)  +  4>,X,  +  y,d,  +  -jTji, 
where  t>,  now  denotes  the  value  defined  by  the  recursion  (for  k  -  N  -  1,  . . . ,  0) 

,  .  /*P***+t 

m  -  n*+t  +  *+t«*  +  .  .  , — ;  tj  -  o. 

®*  +  s*+i  * 


(132) 

(133) 

(134) 


(135) 


(136) 


To  allow  for  later  modifications,  this  is  taken  as  the  definition  of  J,  only  for  (Z„  YHl)  €  Gh  in  which 
case  the  expectations  in  Eq.  (125)  for  V,  certainly  exist.  Hence,  we  can  also  define  the  error  functions 
«,  for  such  (Z„y,_i)  as 

«,(Z„  y_,)  -  V,(Z„  K,_j)  -  J,(Z„  Y,-x).  /-O . N.  (137) 

The  objective  in  this  section  is  to  show  that,  except  possibly  for  the  unlikely  set  of  realizations 
mentioned  earlier,  the  control  u,(.Z,,Y,-X)  differs  from  the  optimal  control  by  less  than  /iV4,  or  more 
precisely,  that  if  there  is  a  control  law  Ux  such  that 

\Uy{Z„Y,.x)  -  u,(Z„Y,-\) I  >  hs/\ 

then  there  exists  a  control  law  U2  €  U  such  that 

|f/2(Z/,y/_,)-«#/(Z„K/_,)|  < 
and 

J(U2)  <  /((/,). 

The  derivation  of  the  result  in  this  latter  form  does  not  require  the  actual  existence  or  uniqueness  of  an 
optimal  control  law. 


5.4  Relations  between  Equation  Systems  I  and  II 

Equation  System  II  canjbe  constructed  if  we  regard  d,  as  one  half  of  v  -  ph  then  expand  Equation 
System  I  (in  the  form  using  A,  and  in  place  ofrj,  and  t>/*)  formally  to  first  order  in  h ,  with  the  quan¬ 
tities  i p,,  X/,  A/,  and  d,  being  of  order  h,  and  finally  relabel  x,  and  X/  as  x,  and  X/  to  distinguish  them 
from  their  exact  values.  If  the  realization  through  an  epoch  /  is  such  that  (Z,f  Y,-x)  €  <7„  therefore,  it 
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is  a  straightforward  but  tedious  matter  to  use  Taylor’s  theorem  with  remainder,  Conditions  (1)  to  (4) 
of  Section  3.1  for  (Z„  T,_  i )  6  Gn  and  the  fact  that  hr 1/16  can  be  made  arbitrarily  large  compared  to 
ln(l//i)  if  we  take  h  sufficiently  small,  to  establish  the  following  result  by  induction  on  the  index  j  of 
these  four  conditions  in  Section  3.1. 

Lemma  11 :  There  exists  an  h*  >  0  depending  only  on  the  parameters  a,  6,  and  F  of  the  unper¬ 
turbed  problem,  such  that  if  h  <  h*  and  (Z„  y,_|)  €  G,  for  some  epoch  /,  then 

U7!  < 

\dj\  <  h'5/l6, 

\xj  -  Xj\  <  Fj  hi1, 

I Pj  +  2 dj  -  Vj\  <  Fjh1, 

and 

|Xy  -  Xy|  <  Fjh1 

for  j  —  0,  ....  /  —I,  where  Fj  grows  slowly  enough  with  j  that  /•}  <  h~l/ 4  if 

j  <  /  < 

The  last  conclusion  here  also  uses  the  inequalities  noted  at  the  beginning  of  Section  4.5.  Further¬ 
more,  applying  the  inequalities  of  Lemma  1  and  the  results  in  Appendix  C  enables  us  to  obtain  the  fol¬ 
lowing  additional  inequalities  during  the  course  of  the  same  induction. 

Lemma  12:  There  exists  an  h*  >  0  whose  value  depends  only  on  the  parameters  a,  b,  and  F, 
such  that  if  h  <  h*  and  if  (Z„  l^-i)  and  (Z/(  Yt-i)'  €  <?,  differ  in  at  most  one  component,  whose 
values  are  denoted  here  as  p  and  p',  then . 

Itfy  <p)  -  Xy(p)l  -  Uy(p')  -  Xj  <p’)J  I  <  Fjh2\p  -  p'l 

and 

|(vy(p)  -  ldj{p)\  -  lvy(p')  -  2dy(p')]|  <  Fj h1  Ip  -  p'l 

for  j  —  0,  . . . ,  /  —  1,  where  Fj  is  the  same  as  in  Lemma  11. 

Again,  the  computational  details  of  the  proof  are  too  lengthy  and  routine  to  be  given  here.  As 
before,  the  notation  x,(p')  is  an  abbreviation  for  XyKZy,  Ty-i)'],  etc.,  where  (Zy,  Ty_i)'  is  the  obvious 
initial  segment  at  epoch  j  of  (Z,,  T/_|)'.  These  results  can  now  be  combined  with  those  of  Lemmas  1 
and  10  to  conclude  the  following  results  for  Equation  System  II: 

Lemma  13:  There  exists  an  h*  >  0  whose  value  depends  only  on  the  parameters  a,  b ,  and  F  of 
the  unperturbed  estimation  problem,  such  that  if  h  <  A*  and  if  (Z,,  y,_,)  and  (Z,.  i ) '  €  G,  differ  in 
at  most  one  component,  whose  values  are  denoted  here  as  p  and  p'  respectively,  and  if 

i  <  -^Vln (l/h),  then 

\e,\  <  -SFTQ  >/9U\(\/h)  <  h~m. 


W  -  p,  -  2d,  |  <  (1  +  Sb>f2)  hv*. 
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lx,  -  0,1  <  (1  +  9. 


»s/?) 


I y,  -  Zp2\  <  b2h  +  6 b(h2/4  +  h2/t)  <  h3/4 


for  either  realization,  and 


IU,(p)  -  ,x((p)]  -  lx,(p')  -  *,(p')]|  <  s/ijb  +  2 cj 

+  J1  +  ^  +  2\f~  a|  ^3Ip  -  p'l. 

I l<r 2 (p )  -  2^(p)]  -  [cr  2(p')  -  2rfj(p')ll  <  24(1  +  >/2C,)A3 

+  (1  +  /)•  +  24-/2  /),)  42|p  —  p'|, 
|d,(p)  -  d,(p')|  <  *3/4  Ip  -  p'l, 


\x,(p)  -  x,(p')|  <  Ip  -  p'|. 

Proof :  To  establish  the  first  inequality,  f-\x^x  is  added  and  subtracted  from  the  definition  of  e,  to 
give  (for  <  >  1) 

e,  -  (z,  ~  fj-iXi-i  ~  1)  ~  -  x,-i). 

Thus,  from  the  triangle  inequality  and  Eq.  (7), 

kl  <  hi  -  *;1  +  l/-ilUf-i  -  x,-,|. 

Since  (Z„  T(_,)  or  (Z„  T(_,)'  €  G„ 

U/  -  x-\  <  V"1;  +  "'7  -s/8 ln(l/ 4)  <  y/b  +  Q  V8  In (1/4)' 

by  definition  and  |x,_i  -  x,_i  I  <  4 7/8  by  Lemma  11  for  sufficiently  small  positive  h.  So,  since  l/  l  <  1 
and  hvs  <  Jib  +  Q)[n(\/h]  for  sufficiently  small  positive  h. 


\e,\  <  Jb+Q 


for  sufficiently  small  positive  h. 


The  next  four  inequalities  can  clearly  be  verified  from  the  triangle  inequality  and  the  results  of 
Lemmas  10  and  11  by  similar  constructions,  Likewise,  the  next  two  (Lipschitz)  inequa’ities  of  this 
lemma  follow  from  Lemmas  10  and  12;  and  the  next  to  the  last  inequality  from  Lemmas  1  and  12, 
since  T,  <  h' 1/4  by  the  recursion  for  T,  and  the  remarks  at  the  beginning  of  Section  4.5. 

For  the  last  inequality,  it  follows  from  Eq.  (127)  that,  for  k  -  0 . /  —  1, 

•**+i  ~  -**+1  ■*  f(xk  —  xk)  +  — —  ( e  —  e')  +  — — —  ■  (ve  —  v'e’)  +  -^-(e2  —  e2), 

P  +  r  (p  +  r)2  r 2 

where  ^’denotes  xk[(Zk,  .y*_|)’],  etc.,  and  where  the  obvious  subscripts  of  the  other  quantities  are 
omitted.  From  the  definition  of  ek+\. 


Xk  + 1  -  xk  -  ( Xk  -  xk’)  +  -^-r  (z*  +  i  -  z*+i') 
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(M  +  rY 


l»(f  —  e')  +  e'Cv  —  «/'))  +  — j(e  +  e')  (e  —  e'). 


Repeated  application  of  the  triangle  inequality  and  the  inequalities  already  established  show  that  the  last 
two  terms  in  this  expression  have  magnitudes  less  than  -JTi  Ip  -  p'l  for  sufficiently  small  positive  h  if 
\xk  -  <  Ip  -  p'l,  since  (Zk.  K*  +  1)  and  (Z*.  >*_i)  €  Gk.  Hence,  again  by  the  triangle  inequality, 

if  \xk  -  x*'l  <  |p  -  p’l,  then 


<\i  r 

+  2  yfii 

\F  +  r 

Ip  -  p’l  if  p  does  not  denote  z*+| 


and 


\xk+i  ~  xt+1| 

Since  |/|  <  1.  p  >  0,  and  r  >  0, 


H  +  r 


+  2  yfh 


Ip  -  p’l  ifp  denotes  zk+ \. 
I-Xa+i  -  **+i  I  <  Ip  -  P'l 


in  either  case  for  all  h  <  /»,  where  h  is  some  strictly  positive  number  not  depending  on  the  index  k. 
Since  the  desired  inequality  is  trivially  true  for  k  =  0,  it  holds  for  all  k  <  /  —  1  by  induction.  □ 


5.S  Some  Loose  Bounds 

Before  proceeding  further,  it  is  helpful  to  establish  some  relatively  loose  bounds  on  the  value 
function  and,  wh^n  it  exists,  the  optimal  control  at  a  generic  epoch  i. 

Lower  Bound  on  Value  Function 


If  the  problem  is  altered  so  that  the  controller  is  given  exact  knowledge  of  the  current  state  as 
well  as  the  noisy  measurements  thereof  at  and  after  epoch  /+1  and  is  allowed  to  use  this  extra  data  in 
the  control  law  in  a  Borel-measurable  way,  this  can  only  expand  the  class  of  admissible  cr  ntrol  laws. 
Hence,  minimizing  over  this  expanded  class  cannot  raise  the  conditional  expected  cost-to-go  at  epoch  /. 


For  epoch  k  >  /  +  1,  define  Hk  ( Zk ,  Yk-\,  x,+  | . xk)  as  the  optimal  expected  cost-to-go  at 

epoch  k  in  the  altered  problem,  i.e., 


Hk  (Zk.  K*_j,  *,+1 . xk)  =  inf  £  ~  s„x,j  +  £  a,x/  +  b,u/ 


where  u ,  is  generated  from  Z,,  F,_|,  x,+1 . x,  by  the  control  law  U ,  the  expectation  is  conditioned 

on  the  same  data,  and  f  2  £  is  the  new  class  of  admissible  control  laws  just  described.  From  the  form 
of  the  dynamics  of  Eq.  (1),  Hk  depends  only  on  xk.  Now  suppose  that 


Hk*\  =  J  gxk\ i  +  axk+ 1  +  y/3. 

Then,  by  the  standard  principle  of  optimality  of  dynamic  programming. 


Hk  +  ]  -  inf  E, 


Is 


-j(ax2+bu2)  +  -jjelA'  +  m  +  (1  +ti/v)w]3 


+  a  [fx  +  u  +  (1  +t/»x)»v]  +  — 
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where  the  k  subscripts  have  been  suppressed.  Evaluating  the  expectations  gives 

Hk  -  infj-^oa^+au2)  +  ygl(/x  +  u)2  +  $(1  +  4>x)2]  +ar(/5r  +  «)  +  y/sj. 
The  infimum  can  be  attained  by  the  use  of 


*+g 


which  gives  (with  additional  k  subscripts  deleted) 

Hktx)  -  -^lax2  +'g/2x1  +  gg(l+tM)2-  -  +  0)  +  afx 

2  0  +g 

Since  HN  (xN)  -  y  sNxj$  for  /  -  iV— 1,  it  follows  by  reverse  induction  on  /  that 

Hi+ito  -  y  g/+i  *2  +  a<+i  *  +  yfl«.| , 
where  gM,  aj+1,  and  /3,+i  are  determined  by  the  reverse  recursions 

A  “  4^  + +  g*+i  gw  “  *w'. 

°*  *  ~b^+~g^[  +  **+<  fl*  "  0; 


0*  “  0*+i  +  g*+i  A  ~  ^ +^+1  ’ 

For  a  general  14  followed  thereafter  by  use  of  an  optimal  control  law,  a  lower  bound  on  the  conditional 
expected  cost-to-go  at  epoch  /  is  therefore 

y[a,«/2  +  a,(x,2+<r/2)}  +  E[J//+i(x,+i)  /Z„  U^\  •  u, J. 

Suppressing  i  subscripts,  this  expectation  is 

^yg<+l  lA  +  «  +  (l+^*)wl2  +  «/+|C/x  +  M  +  (l+d»x)wJ  +  y /9/+| /z,,t/,|. 

Evaluating  this  expectation  as  the  composition  of  a  marginal  and  a  conditional  given  x,  we  get 
y  ft+l  l(/x  +  u)2  +  ?(l+i|»x)2  +  (J*  +  <lt2)<r2]  +  y,+J  (Jx+u)  +  y0/+1. 

This  conditional  expected  cost-to-go  can  be  rewritten  as 

y (i, +*,+,)<«,- «()2  +  |ft(x,2  +  <r,2)  +  <x,x,  +  | 


where 


U‘  f>,  +  g,+ j 

and  ai,Pi,gh  and  gi+\  are  given  by  Eqs.  (138)  to  (140). 
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By  construction,  this  is  a  lower  bound  on  the  conditional  expected  cost-to-go  at  epoch  i  in  the  original 
problem  for  any  control  uh  even  if  it  is  followed  by  use  of  an  optimal  control  law  at  epochs 
/  +  1.  Also,  by  construction  and  the  definition  of  the  value  function  Vh  ut  can  be  chosen  as  u, 

(  a  function  of  Z,  and  YH |,  actually  )  to  give 

K(Zh  YHl)  >  +  <rf)  +  « ,Xt  +  j  |/9,  +  (M2) 


for  all  (Z„  YHl)€Gh  where  xt  and  a}  are  the  conditional  mean  and  variance  of  xt  defined  earlier,  which 
have  been  shown  to  exist  for  realizations  such  that  (ZhYHl)€Gi,  where  Y,- \  is  now  the  sequence 
{50(Zo),5,(Z,.  Y0).  . . . (Z,_,.  Yt^)\. 


Upper  Bound  on  Value  Function 

Returning  to  the  original  control  problem,  we  now  consider  the  obviously  admissible  control  law 
which  gives 


—uk  (as  defined  in  Eq.  131);  A  <  q 
«*  »f  k  -  i 
Sk+lfkZk  +  «fc+l 


At  +  &+t 

and,  for  k  >  /  +  1,  denote  the  corresponding  conditional  expected  cost-to-go  by 


r*(Z*,y*_,)  -  E  .jw?  +  £  Uajxf  +  bjUj2)/ Zk,  Yk-i 
*  J-k  1 


when  (Zk,  F*_,)  is  such  that  this  expectation  exists.  For  *  >  /  +  1, 

»k  -  -CkOfk  +  /%)  -  8*. 

with 


At 


8k+Jk 

At  +  tk+  1 


and 


k  " 


<**+ 1 


(143) 


*  8*  +  «t+i  ’ 

so,  dropping  obvious  subscripts, 

Xft+i  *  if  -  c)  xk  -  8  -  cn  +  (1  +  &x)  w. 

For  k  ^  I  +  l  and  (ZhY,- j)  €  Glt  let  xk  temporarily  denote  E(xk/Zly  YH t)  under  the  control  law  of 
Eq.  (143).  By  the  statistical  independence  of  the  process  and  measurement  noise  variables, 

E(xk^Z„xk)  -  (/  -  c)  -  8, 

so 

E(xk+l/Zl)  -  [f  -  c)  xk  -  8  -  *+,.  (144) 

where  the  dependence  on  x  is  suppressed  in  the  notation.  Also, 

£(x*2+l/**,Z*)  -  if-  c)2x*2  -  28(/-  c)xk  +  82  +  c2rk  +  (1  +  <M*)J4it. 
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so  from  Eq.  (144)  and  noise  independence 

W+i  *  l<y  —  c)2  +  4»2fll  <rl  +  c2r(  1  +  +xk)2q,  (145) 

where  o-|  now  denotes  the  conditional  variance  of  xk  given  Zk. 

By  definition, 

Tk(Z„YHX)  -  TkM,Yt-\)  +  (a***2  +  f**k(x*  +  M+M2) 

for  k  >  i  +  1,  or  with  notational  abbreviations, 

Tk  -  1  +  y{a*(**  +  <rl)  +  M<*2(**  +  W  +  r*)  +  2c*8*5*  +  »|1).  (146) 

Now  suppose  that 

7*+i  “  y £*+i(x*+i  +  <rj!+i)  +  a*+i**+i  +  y(B*+i  +  £*+i)>  (147) 


where  &+i»  «*+i.  and/9*+i  are  given  by  Eqs.  (138)  to  (140)  and  f*+j  is  given  by  the  recursion 

Plr?L ■;  048) 

bj  +  fy-t-l 

Since  this  supposition  is  true  at  k  —  S  -  1,  f*  is  given  by  Eq.  (147)  for  all  ft  >  /  +  1,  as  can  be 
shown  by  backwards  induction  on  k  with  the  use  of  Eq.  (146). 


The  conditional  expected  cost-to-go  at  epoch  i  using  an  arbitrary  u,,  but  the  control  law  of  Eq. 
(143)  thereafter,  is  therefore 

^/z(.r,_,|y  (®/*i2  +  b,Uf2)  +  y  gj+iCx<+ 1  +  <r«.i)  +  «/+i*/+i  +  y  (fli+i  +  (<+i) 

-  y  I«/(*f2  +  <rf)  +  b,u?)  +  y*+|(*+ 1  +  <r2+i)  +  o/+  ,5/+1  +  y  <£/+>  +  f  ,+i)  (149) 

by  idempotence  of  expectation,  at  least  for  iZhY,-\)  €  G,  and  sufficiently  small  positive  h,  when  these 
moments  are  known  to  exist.  Also, 

*+i  “  to  +  «)+(!+  #(*j)  W„ 


so 


^+i  -  to  +  u, 


and 


<r,2+i  -  if?  +  q/bf)<r}  +  (1  +  +ix,)2q„ 

where  x,  and  <r2  now  have  their  former  meaning.  By  definition,  this  conditional  expected  cost-to-go  is 
an  upper  bound  on  V,{Z„YH ()  for  any  u,.  Using  Eq.  (141)  therefore  gives,  for  sufficiently  small  posi¬ 
tive  h. 


V,(Z„Yi.i)  <  y(6,  +  j)  (u,  -  u,)  +  yg,(x,2  +  <T2)  +  a,X, 


1 


fiW+i<r! 


+  tIOi  +  «i*t  + 


b,  +  ft+j 


(150) 


for  all  (Z/,  y,_|)  €  Gh  where  x,  and  rr}  are  functions  of  ( Z„  YHi),  and  /  <  N  -  1. 


SS 
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Limits  on  Optimal  Control 

With  u,  -  u,  in  the  upper  bound  (ISO),  the  principle  oT  optimality  and  the  lower  bound  on  Vt 
preceding  Eq.  (141)  imply  that  u,  —  u  can  be  bettered  (in  the  aenae  of  achieving  a  lower  conditional 
expected  coet-to-go)  unleaa 

(*,  +  gf+i)  («  -  a,)2  < 

at  least  for  (Zh  T,_|)  €  G,  and  for  sufficiently  small  positive  A.  Since 

.  fe*  fktkKi'k 

1/+1  "  ^  t  i  B 

this  condition  implies  that  (w  -  it,)2  <  y  ln(l/A)  for  sufficiently  small  positive  A  and 

N  <  -y>/ln(l/A)  if  the  recursion  (138)  for  gk  is  backward  stable.  This  is  clearly  true  for  small 
enough  positive  A,  since 


&  "  ft  ~r~zh —  +  ft+* +  ft- 
o*  +  &+1 


Also,  comparing  these  parameters  with  the  J  (Z,,Ui-  j)  parameters  gives 

.  -v  [  fkbk  |(  fkh  I  .  v  .  _  .  .  : 


(ft-5*)”  TTT —  ■  —  (&+i -  s*+i)  +  &+i  9k  (*v-%)- 0 

Ok  +  £*+ 1  Ok  +  S*+l 


(so  gk  ^  sk  >  0)  and 


(y*  ~  ^*)  ■  |  bk  +  gi(+i  |  ^*+l  — 

+  [9*  **  ~  (bk  +  gti?(bk  +  *♦,)  (*+,“*+,) 


~  l~ — Z7 — I  ft  'bk  Jk+i ;  < jn  -  4>n)  ”  0. 

J  "**+1  /*+1  j 

Hence,  for  sufficiently  small  positive  h,  |g,+i  -  sB+]|  <  Ayz  and  |y/+]  -  4>t-\\  <  >/A.  Using  u,  to 
denote  the  apparent  approximately  optimal  control  as  before,  i.e., 

«  _  _  ft  S/+1  */  +  4>l+l 

1  b,  +  */+« 

it  follows  easily  that,  for  small  enough  h  >  0, 

|«f  —  5j  <  A-,/2 1  x,  —  x,|  +  AV2  x,  +  yfh. 

From  Lemma  10,  therefore, 

lu,  -  5,|  <  2h  +  VA  <  2>/A  if  |x, |  <  and  (Z„  T,_,)  €  G„ 

or  equivalently  for  sufficiently  small  positive  A,  by  the  definition  of  G,.  This  means  that  under  these 
conditions  u,  cannot  be  optimal  (i.e.,  can  be  bettered  by  some  other  value)  unless 

|u,  -  u,|  <  Vln  (1/A). 
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The  preceding  results  can  be  summarized  as  follows. 

Lemma  14:  For  any  c  >  0,  there  exists  an  h*  >  0  such  that  if  h  <  h*,  if  the  number  N  of 
epochs  in  the  control  problem  of  Section  5.1  is  such  that 

N  <  -L-  Vln  (l/h), 

and  if  (Z,  Y,- t)  €  G,  for  some  /€  {0, 1,  ....  M,  then 

0  <  V(Zh  Y,.x)  -\gt  (jc?  +  <r})  +  a  A  +  y  (/*,  +  *  (M. 

where  S,  (Zt,  Y,- 1)  and  a}  (Zlt  K/_i)  are  the  conditional  mean  and  variance  of  x,  given  Z,,  and  where  g,, 
alt  Pi,  and  ( ,+j  are  defined  by  Eqs.  (138)  to  (140)  and  (148).  Also,  for  any  U  €  (,  if 

I u(z„yhx)-u,  ( z„y,_,)|  >  c-JuTWH), 

then  there  exists  a  u  €  R  such  that  Wt  ( Zh  Y,.h  u)  <  W,  {Z„  K(_,,  U(Z,,  T,.,)!.  where  the 
expected  cost-to-go  W,  is  defined  as 

W,  (Z„  Y,-i.tf)  -  ME/zi  Yi  I  |^(^x„2  +  a,x,2  +  b,u2)  +  ^  y  <«***  +  M*2  )J. 

with  «*  -  U  (Z*,  Yk-X)  for  k  -  /  +  1 . N  -  l. 

Bound  On  |«/+l| 


At  this  point  it  is  convenient  to  extend  the  definition  of  Jl+X  for  all  arguments  (Z/+l,  Y,)  for  which 
the  initial  segment  (Zt,  Y,-x)  €  G,.  If  (Z,+1,  Y,)  9  G(+),  7,+i  is  defined  as 

Jm  (Zl+l,  Yt)  -  ijj  s$  xj  +  j  +  bkuk2)/Zl+ „  T#|,  (151) 

where 


"* 


/fcft+1  Tfc  4-  Otfc+i 
At  +  &+I 


for  Ar  >  i  +  1. 


(152) 


Using  the  result  of  Eq.  (105)  in  the  derivation  of  Eq.  (149)  shows  that  this  extension  of  7/+i  is  well 
defined.  In  this  case,  J,+x  is  defined  as  the  conditional  expected  cost-to-go  using  an  admissible  but  pos¬ 
sibly  nonoptimal  control  law,  namely  the  one  used  to  establish  Eq.  (149).  Hence,  F/+1  is  well  defined 
by  Eq.  (125)  for  such  (Zj+1.  Yt)  9  Gt,  and  the  definition  of  e(+)  can  be  extended  to  this  case  as 


(Z/+ 1,  Y,)  -  K/+1  (Z/+|,  Yt)  -  J(Zm,  Yi).  (153) 

Since  Eq.  (149)  is  valid  in  this  case  as  well, 


0  <  /#+ 1  (Zi+\,Yi)  —  K/+)  (Z'+uY^)  ”  —  e(+i  (Zi+i, Yt)  <  (154) 

for  sufficiently  small  positive  h  and  any  /  -  0 . N  -  1 .  If  N  Vin  (l/h),  (Z,.  T,_j)  €  Gt  and 

(Z/+l,  Y^  9  Gi+ j.  In  this  case,  therefore, 

l«,+i  (Z,+1.T,)|  <  ^ln(l/fi)  (155) 


for  sufficiently  small  positive  h. 
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5.6  Principle  of  Optimality  for  Generic  Epoch  I 

Applying  the  standard  principle  of  optimality  of  dynamic  programming  to  the  optimal  value  func¬ 
tion  of  Eq.  (125),  we  get 

ViiZj,  K,-,)  -  inf  fj-jCa,*/2  +  b,u2)  +  Vl+l[Z,  *  zi+](u),  y,_i  *  u]j, 

where  the  expectation  is  over  xh  wh  and  n/+1  and  is  conditioned  on  Z/  and  YH 1  •  u,  and  where  z,+l(u) 
denotes  the  quantity  /,*,  +  u  +  (1  +  0,x()  w,  +  n/+l. 

Furthermore,  if  (Z,,  K^i)  €  G,  the  expectation  in  this  expression  exists  and  Eq.  (153)  can  be 
used  to  give 


W,.  l/-i)-inf  E  |7+i(Z,  *2/+|(u),  K,_!  *  u] 

+  «/+ilZ/  •  Zj+i(u),  *  «1  +  -j(a,x,2  +  b/M2)J.  (156) 

Also,  the  infimum  in  Eq.  (156)  can  be  restricted  to  the  set 

A'CZ,,  r,_,)  -  { u :  I u  -  fi,(Z,.K,_,)|  <  VTnH7AT}. 
in  this  case  by  virtue  of  Lemma  14,  which,  since  (Z,.  YHX)  C  G,,  is  contained  in  the  set 

{u:|u|  <  2  ln(l/A)) 

for  sufficiently  small  positive  6,  by  Eq.  (131). 

If  7,(Z,.  K/_i)  is  defined  as  in  Eq.  (135)  for  all  (Z,,  K,_|),  then  it  coincides  with  Jt  if 
(Zjf  X,-,)  €  G(.  For  (Z(<  y(_,)  €  G,  but  (Z/+1,  Y,)9  1,  however, 

7+1  —  7+i  ™  "J  l&+i(*i+i  +  crh\)  —  s,+l(x<+ 1  +  pl+x  +  24+1))  +  a(+1  x(+| 

“  ^1+ 1  *1+1  ~  ^(+1  7+1  +  \  Pt+i  +  f/+ 2  +  ‘V  —  ■*>/+!  •  (157) 

*■  6/+1  +  &+ 2 

It  is  straightforward  to  verify  that  (Z„  K,-,)  €  G,  and  !«,  -  fi,(Z(.  y,_|)|  <  Vln(l/6)  imply  that 
(Z/+i,  Yt)  €  Gy+j  (Z1+1  —  xl+x)2  <  8(m,+1  +  r/+|)  ln(l/6). 

Therefore,  Eq.  (156)  can  be  replaced,  for  (Z,,  y,_i)  €  G,  and  for  all  sufficiently  small  positive  h,  by 


V(Zt,  y<-i)*  a  inf  £I7+i(Z(  *  Z/+i,  Yt-\  •  u) 

SK4/VZ/. y 

+  «,+i(Z,  •  2/+i,  y,_  1  •  a)  +  -j(a,x>2  +  7«2)1 


+  X  (7+1  (Z/  •  P-  F/-1  •  «)-  7+,(Z,  *  p,  r,_, .  a)]  dPZi+l(p) |.  (158) 

where  the  first  expectation  and  the  probability  measure  Pz  are  conditional  on  Z,,  T,.),  and  a  as  in  Eq. 
(156),  and 

/  -  [p:(p  -  x*+i  )2  >  8(m,+i  +  r,+i)  in(l/A)}. 
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It  is  a  fairly  straightforward  but  tedious  matter  to  conclude  that  the  integral  term  in  Eq.  (158)  is  smaller 
in  magnitude  than  A3  for  any  u  €  A,(Zlt  and  sufficiently  small  positive  A  by  the  use  of  Eq.  (157), 
previously  established  error  bounds,  and  inequality  (105)  for  large  values  of  Ip  -  x'+i  |.  As  a  result,  it 
follows  from  further  straightforward  manipulation  that 

K(Zh  YHl)  -  Pi(Z„  K,_|)  +  P2(i)  +  Pi(Zj,  y,_,) 

+  P4(Z/.  y/-.)+  inf  H0(Zit  Y^.u)  (159) 

for  (Zj,  Y/—i)  €  G,,  A  sufficiently  small,/  <  N  —  1  <  -^-Vln(l/A), 


where 


*/ 2(a<  +  fi2sl+l)  +  i  rji+i  +  s,+ip(+l  +  ( ftp,  +  q,  +  rj+1)  I”-! 

+ + + l^f) 


+  (V/+ 1  +  S/+i)  - — —  9(0/ 

Pi+ i  ■+■  n+i 


■/mL.+w  +M1, 

M/+1  +  f/+l  M/+1  +  r/+l  (  T/+i  J  j 


and  where  P2,  Pj,  and  P4  are  rather  complicated  expressions,  for  which  it  follows  from  the  inequalities 
of  Lemmas  11  to  13,  the  results  of  Appendix  C,  and  the  backwards  stability  of  Eq.  (132)  that  P2  is 
independent  of  (Z/f  y,_i), 

|P2|  <  hVA,  |/»3|  <  AV2.  (161) 

and,  if  (Z,.  Yi-\)'  €  G,  differs  from  (Z„  K/_j)  in  only  one  component,  whose  values  in  (Z/(  ()  and 

{Zj.  Y,-i)‘  are  denoted  by  p  and  p'  respectively, 

\P4(p)  -  P4(p')\  <  A"7/,‘(A3  +  A2 Ip  -  p'l),  (162) 


and  where 


Hq(Zi,  J/_|,  u )  ™  y(A,  +  S/+ j)  if2  +  Ist+tf/X,  +  0/+|  +  C(Zt,  J/-i)]» 

+  g<(Z/,  K/_i,  «), 


+  ~-{/(2lo’2  ~  Pi  -  (*i  -  x,)21  +  9(0/12^  +0/(x(2  +  a-,2))) 
*i+t 


8/(Z/,  Yh\,  m)  —  ff/z(.K(_,.l»l«/+i(2/  *  */+i(«).  I/— i  *  «f)l-  (165) 

Applying  the  inequalities  of  Lemma  13  and  the  results  in  Appendix  C  on  composite  Lipschitz  condi¬ 
tions  to  Eq.  (164)  we  can  show,  after  some  routine  manipulation,  that  for  sufficiently  small  positive  A, 

|C(Z/,J'/_,)|  <  Ay2  (166) 
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and 

lc[(zt  r,_,)i  -  c[(z„  r#_i)TI  <  a-7/,6(/»3  + /»2lp -p'l)  (167) 

if  (Z„  K/_i)  and  (Z„  are  both  in  G,  and  differ  in  only  one  component,  whose  values  are  denoted  p 
and  p'  respectively. 

From  this  point  on  it  is  assumed  for  simplicity  that  there  exists  a  Borel-measurable  uq(Z(,  F,_ i )  for 
all  (Z/(  Yi-i),  i  -  0 . N  —  1,  such  that 

-  inf  H0(Zi,YHl.u). 

U 


If  such  a  minimizing  «*  does  not  always  exist,  the  ensuing  results  can  still  be  obtained  by  a  standard 
limiting  procedure.  With  this  assumption  and  the  definition 


ut(Zt.YHl)~ 


WtW.n-i)  +  »,+i  +  c(z„r,_,) 

b,  +  s(+1 


(168) 


it  follows  with  considerable  computation  from  Eqs.  (135),  (137),  (159),  and  (163)  that 

«/(z<Iy/-,)  -  p2d)  +  PAZi.Y^)  +  p4(z,.y,- ,)  +  j  (j/+i  +  */)(«;-  «,)2 


s;+l//x/(z/,y<-))c(z/,y<_i) 

s/+i  +  6/ 


c(z(.y(_,) 


*,+,+  yC(z„r,_i> 


S,+1  + 


+  8,[z(,  Y,-\,u'(.Zi,  K/-,)]  (169) 

for  all  €  G,  . 

It  is  also  convenient  to  establish  the  following  results  for  future  use. 

Lemma  15: There  exist  an  h’  >  0  and  a  f  >  0  ( independent  of  h  and  epoch  index  ft  such  that  if 


h  <  h\ 

i  <  N  -  1  <  -^y/\n(l/h), 
(Z„F,_,)  €  G„ 
|x,(Z„K,_i)|< 

| d^.Y'-O  <>tKh, 
u, m,YHl) 


<  Vlnd/A), 

|  Z/+ 1  —  x,+1  (Z/,  Yt-\  •  U()J  ^  8(w,+i  +  rl+i)\n(l/h). 


and 


NRL  REPORT  8406 


*/+ i(Zi  •  z/+i>  Yi-i*  ut  ^  K 


•  M;  ^  Kh. 


Proof:  This  is  a  routine  matter  of  substituting  inequalities  already  established  for  the  various  vari¬ 
ables  and  the  definition  of  Gt  into  Equation  Systems  I  and  II  and  using  the  triangle  inequality.  The 
desired  result  then  follows  from  the  fact  that  h~*  >  ln(l//r)  for  sufficiently  small  h  if «  >  0.  □ 

Lemma  16:  There  exists  an  h‘  >  0  such  that  if  h  <  h\  i  <  N  -  1  <  -jp/lnO/h),  and 
(Z,,  T,_,)  and  (Z,,  YH\Y  €  G,  differ  in  only  one  component,  whose  respective  values  are  p  and  p',  then 

I  fi,(p)  -  u,(p')|  <  hs/i  +  Ip  -  p'l. 


Proof:  Since  |/|  and  -  —  are  both  strictly  less  than  unity,  this  lemma  follows  if  we  apply 

b,  +  s/+i 

Lemma  13  and  Eq.  (167)  to  Eq.  (168)  and  use  the  triangle  inequality.  □ 

Lemma  17:  There  exists  an  h'  >  0  such  that  if  h  <  h’,  i  <  N  -  1  <  ^-Vln(l//i),  and 
(Zh  Yj-\)  €  G,  ,  then 

|e/(z/,r/_,)-6>(z/>T>_,)|<  }a5/4. 


Proof:  From  Eqs.  (131),  (168),  and  (166), 


u,  -  uA 


ICl  ^  h*' 

b,  +  s,+,  ^  b,  +  s/+ 1  ’ 


in  this  case  for  sufficiently  small  positive  h.  Since  b,  ^  B  >  0  and  s/+1  >  0,  their  sum  is  less  than 
h~l/*  for  h  <  B~4.  □ 

Definition:  For  h.K  >  0,  /  -  0,  ...  ,N,  is  the  subset  of  (Z,,YHy)  €  Gt(h)  such  that 

\xj(Zj.Yj.i) I  <  K 

and 

I dj(Zj,Yj-i>\  <  Kh 

for  y-0,  ... where  (Zj,  Yj-\)  denotes  the  indicated  initial  segment  of  (Z,,  T/_|). 

The  control  law  U  is  now  considered  as  an  approximation  to  an  optimal  control  law,  where 

«,  if  {Zh  Y,-\)  €  ir,(h_,/'6) 

U (Z/t  Y/_j)  —  fgi-i 2/ +  a i+i 


b,  +  ft+i 


otherwise. 
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It  is  shown  next  in  Theorem  2  that  (Z,,  remains  in  nl(h~'/ib)  with  high  prior  probability  for  small 
A,  it  will  be  shown  in  Section  S.7  that  ut  is  a  first-order  approximation  to  an  optimal  control  in  this  case. 


Theorem  2:  If  the  total  number  of  epochs  N  always  satisfies  the  inequality 

N  <  Vln  ln(l/A) 


and  the  control  law  0  is  used,  then,  a  priori, 

lim  Pr{(Z„  F/_t)  €  irt(h~i/l6)\  i  -  0 . N(h)}  -  1, 

A— 0+ 


assuming  of  course  that  the  parameters  a,  b,  F,  sN,  A,  and  B  of  the  unperturbed  problem  do  not  vary 
with  N. 


Proof:  For  the  problem  at  hand,  let  c  and  h  be  the  positive  constants  cited  in  the  conditions  of 

Section  3.1,  and  assume  that  c  <  2  without  loss  of  generality.  Let  [9t:  i  -  0 . N  -  1}  be  the 

sequence  defined  by 

2 

0,+i  *o-  >  1 


and  let  <o,  denote  In  9t.  Now  suppose  that 


max{U0-*ol.  l*'oi.  I%l.  (|w,|.  |n,|}}  <  c  V«o 


(170) 


and  max{|«/|:>  -  0,  ,  i  -  1)  <  cco/_i  for  some  t,  with  h  <  h.  By  construction, 
1  <  0o  <  0t  <  . . .  .  so  o>0  <  lnd/A),  since  the  logarithm  is  monotonic.  Therefore,  by  the  conditions 
of  Section  3.1  on  the  unperturbed  problem, 


(Zi.  fl_,)  €  G, 


1 

0/-1 


if  A  is  also  chosen  as  less  than  h.  In  light  of  the  previously  established  inequalities,  a  strictly  positive  h\ 
exists  such  that  if  h  <  hu  then 

luj  <  2ln(l/A) 

whenever  (Z*.  JV-i)  €  Gk(h)  and  k  <  -^-VlnO /h).  Hence  if  h  -  is  further  chosen  to  be  less 

51  9  o 

then  A|,  then 

Im/(Z(,  F/_i)|  <  2  In  <  c 


Since  u,  >  <a,_1  by  construction,  it  follows  from  an  obvious  induction  on  /  that 

<z„  rj  e  c*!^] 

whenever  inequality  (170)  holds.  From  its  definition  hi  <  h\  -►  Gs^hi)  2  GN{h\).  So,  since 
9N  >  “  1/A  by  construction, 


(Z/v,  fy-t)  €  G\(A) 

for  all  A  less  than  some  strictly  positive  value  A*,  if  W  <  -jj  >/!n(l/A)  and 

max(|x0-xol,  |w0|,  |n^|,  max  ( I mv I,  In/I)}  <  c\/ln(l/A). 

/"I . A/“l 

Also,  A*  >  0  can  be  chosen  so  that  Lemma  1 1  holds  under  these  conditions,  too,  in  which  case 

(Z/v,  K„_|)  €  irv(A-1/16). 


i 
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From  the  bound  in  Section  3.1,  the  prior  probability  P  of  this  event  not  occurring  is  bounded  above  by 

n  I  | 

p  e  '  ' 


cy/2n  I  Vlny 


where  a1  -  max{v0,  q0,  rN,  max  {q,,  /•,}}  and  y  and  p  denote  \/h  and  — ,  for  brevity.  Since 

. _ _  .  ■  i-l . N-\  C 

N  <  vlnln(l//t),  another  bound  is 


^2  y[^'np-(7| 


cy/lir 


To  establish  that  P  approaches  zero  as  h  does,  as  long  as  N  <  %/ln  ln(l//i),  it  therefore  suffices  to  show 
that  the  limit  of  the  exponential  factor 

tt 


N{y)  lnp 


p~JV<>)  In  y 


is  zero  as  y  — «».  Clearly  N(y)  can  be  taken  as  equal  to  Vln  lny  for  this  purpose.  The  logarithm  of 
the  subtrahend  in  this  quantity  is 


2  In  —  +  In  In  y  -  In  p  Vtn  In  y, 

O’ 


which  can  be  made  arbitrarily  large  for  any  c  and  <r  by  making  y  large  enough.  Also,  the  ratio  of  the 
minuend  to  the  subtrahend  is 

12 


<r  ,  N 

c  ‘np  polity 


The  logarithm  of  the  last  factor  is 


InW  +  N  lnp  -  Iny  <  In  (Vln  Iny)  +  InpVln  lny  -  Iny, 

which  goes  to  -<»  as  y—  Since  the  other  factors  are  constant,  the  entire  ratio  goes  to  zero  as 
y  —  Therefore, 

lim  P(h)  -  0. 

*~o 

Since  (ZN,  Yn-\)  €  SN(h~l/l6)  +  ( Zh  YHi)  €  n,(h~ 1/l6)  for  all  /'  €  {0,  ... ,  vV),  by  construction, 
the  theorem  follows.  □ 

5.7  Induction  Argument 

In  this  section,  an  induction  argument  is  developed  which  demonstrates  as  a  corollary  that  the 
control  u,  generated  by  control  law  0  is  optimal  to  order  h  for  realizations  in  it,  (/T1/l6)  for  sufficiently 
small  h.  For  a  generic  epoch  /  <  N  -  1,  the  induction  hypothesis  is  that,  for  some  h*  >  0  which  does 
not  depend  on  the  epoch  index  /,  if 

h  <  h*. 

JV  <  -i-  VlnO/fj), 

<?,+,  €  [A-V*  h~xlt], 

and  (Z,+ 1,  T,),  (Z,+l  Y,Y  €  ir,+l  (Q,+i)  differ  in  only  one  component,  whose  values  are  denoted  as  p 
andp'  respectively, 
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then 

|«,+1  <p )  -  «,+,  (pOI  <  Km  h~vl  (A3  +h2\p-  p'l);  A" 1/16  <  KM  <  A"*/*. 

Here,  «/+i(p)  is  shorthand  for  «/+1  l(Z,+ 1(  )/)],  etc. 

Lemma  18:  If  the  preceding  induction  hypothesis  holds,  then  there  exists  an  A  >  0  (independent 
of  A*)  such  that  if  A  <  A,  -  f  Qi+ 1,  f  as  in  Lemma  15,  (Z,.  Y,-i)  and  (Z„  K,-!)'  €  irt  (Q,)  and 
differ  in  only  one  component,  whose  respective  values  are  p  andp'  respectively,  and  if 

I «/  -  Hi  ip) I,  In/  -  «,</>')  |  <  Vln(l/A)( 

then 

|#(  (p.  «,)  -  8,<p\  u/)l<  j|  +  4*(+2  A- 1/3  +  in2(l/ A)  C,+,Ja3 

+  Kl+i  Ayi  |u  -  n'l  +  (2 h~m  +  lnJ(l/A)Z>/+i)AJ  Ip  -  p’l. 


Proof:  Let 


and 


z<+ 1  ~  *1 
yJmM  (p)  +  r,+1 


*t  ”  //i/(p)  +  u , 


By  definition,  then 
8/  (p.u,)  -  8,  (p'.ui) 


h  “ 


2j+i  ~  22 

y/mn\(p')  +  r,+ 1 


z2-/(i((p')  +  «/. 


E  |«<+,  [Z,(p')  •  (z,  +  (p)  +  r(+1),  T,_,  (p)  •  i/JJ 

—  £  U/+i  [Z/(p')  •  (z2  +  t2-Jml+ 1  (pO  +  0+1), K/-!  (p')  •  u,  1). 


By  construction,  fj  -  s(p)  and  /2  —  s(p’)  in  the  context  of  Section  4.4,  so  by  Eq.  (96)  and  integration 
of  Eq.  (95)  over  Ul  <  c M,  -  6\/ln(l/A), 

Prd^l  >  V8  ln(l/A)J  <  \  J-  L  2; 


for  sufficiently  small  A  >  0  (because  M,  >  jA_1/®),  this  probability  being  conditioned  on  (Z„  y,_i)  € 

G,  (A)  and  on  u,.  This  result  does  not  depend  on  any  restrictions  on  u(.  For  sufficiently  small  positive 
A,  therefore,  given  such  (Z/,K/_ 2), 

Prlltil  V«(+i^»)  +  O+i  >  8v'In(l/ A)}  <  -  ^yhy 

and 

Pr||/2|  V m(+|(p')  +  rl+\  >  8Vln(l/A)}  <  4  |n  (i//j)  ’ 
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By  inequality  (155),  |«/+1|  <  -jln(l//t)  in  such  cases  (for  sufficiently  small  positive  h).  Since  the  con 

ditional  density  of  t\  and  t2  exist  otherwise,  it  follows  from  the  triangle  inequality  and  the  use  of  t  to 
denote  both  variables  of  integration  that 

y«ln(l /h) 

|8(p,  u )  -  8 (p ',«')!  <  y  hl  +  k+i  •  (*i  +  h/m  +  r ),  Y  .  u (r) 

-«,+i  IZ'  .  ( z2  +  N/m'+  r),  T  •  u')\p,^_u.  dk„  (171) 


where  some  obvious  epoch  subscripts  are  suppressed  in  the  notation.  By  construction,  p,jP  MU),  j  -  1, 

2,  are  independent  of  the  value  of  u,  so  the  integrand  of  Eq.  (171)  can  be  rewritten  as  the  absolute 
value  of 

«/+]lZ  «  (z,  +  tJm  +  r ),  Y  *  u)\p{t/p)  -  p(t/p')) 

+ P (t/p ’){«,+ , \Z  •  (z,  +  r>/m  +  r).  Y  •  m]  -  €I+)|Z’*  (J2  +  tJm'+  r),  )"*u’]).  (172) 


The  first  term  in  this  expression  is  bounded  in  magnitude  by 


~ln(l//r)(C(+1  k*  +  Di+i  h*\p  —  p'|) 


~y  +  2M 

yfht 


from  inequality  (155),  Lemma  1,  and  the  proof  of  Theorem  1.  By  construction, 

|/il  <  -v/8  ln(lA/f)  ♦  |zj+i  -  x*+i  (p)|  <  ^/HKm.+ifp)  +  rj+1)ln(l/A) 
and 

|f2|  <  >/8  ln(l//i)  ♦  |z,+|  -  x'+x  (p')|  <  V8  ip')  +  rl+il  ln(l//i). 

By  hypothesis  and  Lemma  15,  therefore, 

(Z/(p)  •  z(+|,  Yj-i  (p)  •  u,]  €  nt  ( Q ,) 
and 

[Z,<p'>  •  z(+ 1,  K-i  ip')  •  «/l  €  ir,(Q() 

for  such  z,+|.  Hence,  by  the  induction  hypothesis  and  the  triangle  inequality,  the  other  term  in  (172)  is 
bounded  in  magnitude  by 

p(t/p')Kj+\h~'n  [4/t3  +  h2(\p  -  p'|  +  |z,  -  z2|  +  IrllVm  -  \Tm'\  +  |u  -  u'|)l 
for  sufficiently  small  positive  h.  Also, 

I?!  ~  Z2|  -  1/  U/(p)  -  X,(p')j  +  Uj  -  m/|  <  |x,(p)  -  x,(p')|  +  |m;  -  m/| 


and,  by  the  proof  of  Lemma  1, 

where  ri+i  <  h~i,A  if  i  <  ~yj\n{\/h).  Combining  these  results  and  substituting  the  bound  for  (172) 

in  the  integral  of  inequality  (171)  we  obtain  the  conclusion  of  this  lemma  for  sufficiently  small  positive 
h  after  some  routine  computation.  □ 


I y/mip)  -  Jm(p') |  - 


m(p)  -  m(p') 


yfmTp)  +  ' Jm(p ') 
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Lemma  19:  If  the  preceding  induction  hypothesis  holds  and  (Z,,  J'.,)  €  n ,((),),  with  Q,  as  in 
Lemma  18,  then  there  exists  an  h  >  0  such  that  if  h  <  h 


b,  +  s(+1 


-)  hin 


k*(z,,  yhi)  -  u,(z„  n_,)|  <  4 
for  all  (Zi(  y,_,)  €  n,(Q,). 

/Voo/-  Since  (Z„  )^_i)  €  7r,(f  Q,+1)  -*•  (Z„  y,_|)  €  G,,  |u,  -  u,\  <  J\n(\/h)  and  \u'  -  u,\  < 
Vln(l//i)  by  Lemma  14  if  h*  is  chosen  to  be  less  than  the  value  needed  in  that  lemma.  So  it  follows 
from  Lemma  18  that 


1 8,[«;(Z„  -  8,[«/(Z/.  17-1)11  <  SKi+]h~l/2(h3  +  h2\u’ ~  u:\) 

for  all  sufficiently  small  positive  h. 


(173) 


Now  assume  that  | it'  -  «,  |  > 


fjUti+L.  +  ±\ 

,  V  -S/-M  +  A  J 


-j(s  +  b)  (u*  -  u)2  >  --  (s  +  b) 


10  A  2 


s  +  b  b 


hin.  Then,  dropping  epoch  subscripts, 
h*1  \u'-u\ 


>  {(S  +  b)  !«•  -  fil  +  \  !«•  -  si) 

>  j(s  +  b)  Ay2L +  T  l«*  -  fill 

L  [V  JT#  V  JT#  ft  I 

>  {' <* + » *]  \rri\ + { (H*) 2*v'  i"*- 


>  5Kh~vlUi3  +  A2  |«*-fi|). 

By  inequality  (173)  and  the  triangle  inequality,  therefore,  under  this  assumption 


»,(«,)  -  8,(u,*)  <  j(s/+i  +  b,)  («*-  id,)1. 


Let 


/f,(Z„  F,_„  u)-H0{Z„  I#— i,  «)-8,(Z„  T,_„  u). 

For  the  values  of  Z(  and  K/_  j  under  consideration  here, 

u,  -  arg  min  {//](«/)} 

If 

by  construction.  Also,  for  a  general  u, 

//,<»>  -  4  + m  (« -  *,>>. 


(174) 

(175) 


(176) 


Therefore,  deleting  epoch  subscripts, 

|(s  +  #)  («•  -  fi)J  >  8(«)  -  8(«»)  -►  //,(«•)  +  ~  iSlLt± ±  c>2  +  8(k.)  >  6(i}) 

<  i  s  +  b 

-►  #,(«•)  +  8(«*)  >  -  -5-  &L±£±C)i  +  5(g) 

4  S  t  ft 
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//0(«‘)  >  H0(u)  by  Eqs.  (175)  and  (176)). 


Combining  this  result  with  inequality  (174)  and  Eq.  (159)  shows  that 


k  *~  M,  l  < 


10  Ki+'  +  2-  h  m 

SM  +  b,  b ,  ’ 


because  the  reverse  inequality  implies  that  u*  is  nonoptimal  (strictly  less  optimal  than  S,,  in  fact). 
Since  Ki+i  >  A- 1/16  by  assumption,  the  lemma  follows  for  all  h  less  than  some  sufficiently  small  posi¬ 
tive  value.  □ 

Lemma  20:  If  the  preceding  induction  hypothesis  holds  and  (Zt,  Yt- 1)  €  irfQf,  with  Q,  as  in 
Lemma  18,  then  Eh  >  0,  such  that  if  h  <  A, 

\u;i(Z„  K/_i)l  -  u,*[(Z(,  rj_,)']|  <  h  +  Ip  -  p'l. 

Proof:  By  the  triangle  inequality, 

|l<*(p)  -  U*(p')|  <  | u'(fi)  -  U,(p)\  +  |fi,(p)  -  Ut(p')\  +  |<4> (p’)  ~  M*(p')l- 

The  lemma  then  follows  by  Lemmas  16  and  19.  □ 

Lemma  21:  If  the  preceding  induction  hypothesis  holds  and  (Z,,  ()  and  (Z,,  Yt-iY  €  rr,(Q,)s 

with  Qt  as  in  Lemma  18,  differ  in  only  one  component,  whose  respective  values  are  p  and  p'  respec¬ 
tively,  then 

|8,|p,  i/,*(p)]  -  8, Ip'-  «;<p')II  <  J  +  SKMh~V2  +  ln2|-jjj  CM  AJ 

+  [3tf,+1A-V2  +  ln2(l/A)  Di+l]h2  Ip  -  p'|. 

Proof:  From  Lemma  14,  |w*(p)  -  «/(p)|  <  Vln(l/A)  and 

I  u,*(p')  -  u,(p')l  <  Vln  (1/A) 

for  sufficiently  small  positive  A.  Substituting  u'(p)  for  u  and  u°(p’)  for  u'  in  the  inequality  of  Lemma 
18,  and  using  Lemma  20  and  the  triangle  inequality,  we  can  establish  the  lemma.  □ 

Theorem  3:  There  exists  an  A*  >  0  which  depends  only  on  the  parameters  a,  6,  F,  SN,  A,  and  B 
of  the  unperturbed  problem,  such  that  if 

A  <  A*; 

S  <  -^->/ln(l/A); 

Qi  —  tQi+u  i  “  0,  ....  N  —  1; 

and 

Qn  -  h-u' ; 

where  f  is  the  constant  required  by  Lemma  15,  and  (Z,,  Y,-\)  and  (Zh  Y,- ,)'  €  v,(Qi)  and  differ  in 
only  one  component,  whose  values  are  denoted  here  by  p  and  p'  respectively,  for  any  /  €  (0 ,  ....  M, 


l«(  (p)  —  «,  (pOl  <  K,  A",/J(AJ  +  A2  |p  -  p'|), 
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where  K,  is  determined  by  the  recursion 

K,  -  8 Km\  Kn  -  A-'"6. 


Proof  (reverse  induction  on  i):  By  definition,  given  the  premises 

Jv-i)  ™  y  Sy  E(xn/ Zs,  JV-i) 

-  jsN(x$  +  o-D 

and 

Js(Zn,  W  -  —  sN(xj$  +  pN  4-  2 dN), 
since  (Z/v,  fy-i)  €  n ,(QN)  -►  (Z„,  Ys-\)  €  (7#. 

Thus, 

*n(Zn,  Yn-0  “  "j  */r  K*w  +  xn)(xN  —  x\)  —  p*  +  (<rjv  —  2djv>], 

and  the  conclusion  of  the  theorem  follows  directly  by  application  of  the  triangle  inequality  and  the  in¬ 
equalities  of  Lemmas  11  and  13,  if  A*  is  sufficiently  small. 

If  the  theorem  holds  at  epoch  /+ 1,  then  the  initial  segments  (Z/f  Y,-i)  and  (Z,,  y,_,)’  are  both  in 
G,  (A).  Since  the  maximum  value  of  sufficiently  small  positive  A  required  for  inequalities  (161),  (162), 
(166),  and  (167)  and  those  of  Lemmas  11,  13,  19,  and  21  depend  only  on  the  parameters  a,  b,  F,  and 
B  of  the  unperturbed  problem.  A*  >  0  in  this  theorem  can  always  be  chosen  as  the  minimum  of  these 
values.  Substituting  these  inequalities  into  Eq.  (169)  and  using  the  results  in  Appendix  C  for  Lipschitz 
conditions  of  composite  functions,  we  can  show  that  this  theorem  holds  at  epoch  /,  as  long  as  N  is  small 
enough  that  the  recursion  generating  K,  does  not  make  K,  >  h~v*  for  KN  —  h~V16.  But 

K,  <  *0-  8/vA~1/l6, 

so 

In  A)  <  Mn8  +  In  (1/A)  <  yin  (1/A) 
for  sufficiently  small  A*  >  0  and  A  <  A*,  or 


K,  <  h~vt 

in  exponential  form.  □ 

Corollary:  Given  the  premises  of  the  theorem, 

I«;(Z„  r,_,)  -  tt,(z„  r,_,)|  <  a*4 

if  A*  is  sufficiently  small. 

Proof:  By  the  triangle  inequality, 

I «<*-  fi/l  <  W~  «il  +  l«i  -  «il- 

Since  (Z„  Yt-i)  €  ir,(Q,)  +  (Z„  YHl)  €  G„  the  corollary  follows  by  Lemmas  17  and  19.  □ 


,•  .  ■  i.--  Tyikc. 
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Even  if  a  strictly  optimal  control  law  does  not  exist,  it  can  still  be  similarly  established  via  a  limit¬ 
ing  procedure  that  if  under  the  premises  of  Theorem  3 

\u(zt.  YHt)-uAzh  n_,)l  >  hitA 

for  any  admissible  control  law  U,  then  there  exists  a  u  €  R  such  that 


W,(Z,.  F,-,.  u)  < 


W,  IZ„  Y,.u  UU>. 


where  If'  is  as  defined  in  Lemma  14.  Hence  the  admissible  control  law  U\  constructed  from  U  by 
replacing  U  (Zt,  K,_|)  with  u,(Z,.  F_i),  is  such  that 

JW)  <  JW). 


6.  CONCLUSIONS 

One  is  often  interested  in  systems  which  are  controllable  but  unstable  in  open-loop  operation  (i.e., 
with  u,  =  0).  Strictly  speaking,  this  situation  is  not  covered  here  because  1/1  <  1  in  Eq.  (1).  This  is 
not  really  a  limitation,  however,  because  such  a  problem  could  always  be  reformulated  in  terms  of  devi¬ 
ations  from  a  stabilizing  control  law  of  the  form 

«/“ -c,z,  .  0  <  /-  c,  <  1, 

in  which  case  the  dynamics  of  Eq.  (1)  become 

x,+i  -  (/  -  c,)  X,  +  u,  +  (1  +  Ip,  x()  w,  -  c,  nh 

where 


U,  -  M,  +  C,  Z, 


now  plays  the  role  of  the  control  variable.  The  analysis  could  then  proceed  as  before,  but  with  some 
extra  terms  appearing.  This  extra  generality  was  not  included  because  the  analysis  was  already  very 
complicated  and  was  only  intended  to  be  exploratory. 

Other  interesting  extensions  of  the  results  here,  or  similar  ones,  would  be  to  the  multivariable 
and  continuous-time  contexts  and  to  steady-state  behavior  in  infinite-time  problems.  As  a  first  step 
toward  analyzing  the  continuous-time  case,  one  might  consider  adapting  the  approach  used  here  to  a 
discretized  problem  of  the  type  described  in  Ref.  1,  where  there  is  a  time-increment  parameter  A  (i.e., 
the  time  between  successive  epochs)  which  is  small  compared  to  the  perturbation  parameter  h.  The 
results  obtained  would  at  least  have  a  formal  bearing  on  the  limiting  contnuous-time  problem,  and 
might  be  suggestive  for  conducting  a  mathematically  precise  analysis  of  it.  This  sort  of  procedure, 
however,  seems  to  require  the  use  of  a  third-order  Taylor  series  expansion  in  carrying  out  the  propaga¬ 
tion  step  of  Section  4.3,  and  with  the  sort  of  constructions  used  here,  it  did  not  produce  any  useful 
results  in  the  updating  step  of  Section  4.4. 
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Appendix  A 

NONEXISTENCE  OF  PROBABILITY  DENSITY 


For  simplicity,  we  consider  only  the  particular  case  of  a  single  transition  where 

y  -  x  +  (1  +  hx)ot. 


with  x  and  w  independent  zero-mean  unit-variance  normal  random  variables,  and  show  that  the  proba¬ 
bility  density  of  y  diverges  at  ~l/h.  From  the  constructions  of  Section  4.3  and  their  role  in  the  overall 
estimation  problem,  the  same  reasoning  can  be  applied  to  more  general  examples. 


If  the  new  random  variable  w  is  defined  as 

w  -  (I  +  hx)u», 


then 

and,  except  at  (x,o>)  -  (-1/A0), 


y  —  x  +  w 


e  J  2<l+ta)J 

'<*•■’) ‘ piwM ~  hli  +  tel 


Hence,  for  positive  «  < 


3  h' 


+  «  ^  p(x,w)  dx  dw. 


For  each  value  of  w  in  this  region  of  integration, 

r 

1 


p(x,w)  >\ 


2irh(w 


Since  2«  <  w  <  4-  —  e 

— — 1<  2  and 

w  + 

+  c 

h 

w  -  c  1 

h 

4 

p(x,w)  > 

c 

2  irh 

in  this  region.  Substituting  this  bound  in  the  preceding  integral  and  changing  the  variable  of  integration 
over  w  to 


u  -  w  +  c 


gives 
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Thus 


for  all  positive  c  less  than  I !h.  For  any  fixed  h  >  0,  this  lower  bound  approaches  infinity  as 
Hence,  the  probability  density  function  of  y  diverges  at  -\/h. 


Appendix  B 
SPECIFIC  CASE 


In  the  context  of  Section  3.1,  assume  that  inequality  (13)  holds  with  k—  12,  h  -  e~s,  and  c  — 
~ ;  that  x0  “  0,  and  v0  -  1;  and  that  /,  -  1/2  and  qt  -  o+1  —  1  for  /  -  0, . . . ,  N  -  1.  Assume  that 

h  <  h.  Since  h  <  h, [h  ln(l//>)l  -  ln(l//i)  -  1  >  0,  so 
an 

h  ln(l/A)  <  Se~s  <  0.04. 


M/  1 

Also,  Pt  —  — —  and  m,+i  -  —Pi  +  1.  Hence,  by  an  easy  induction, 
+  l  4 


J1/+1  is  in  the  interval  (1 

Pi  €  (1/2, 1) 

„  _  J _ L 

2k  24 ' 


.  5/4) 

/  -  0 . N  -  1, 


b  -  ~k  -  15. 

4 

Now  assume  that  Conditions  1  through  4  of  Section  3.1  are  met  at  generic  epoch  i  (i.e.,  that  for  this 
realization  (Zt,  Y,-i)  €  G,)  and  also  that  |x,  —  x,  |  <  VlnO/h)  for  this  realization.  We  next  verify  that 
this  implies  the  same  conditions  at  epoch  i  +  1.  This  is  done  in  two  steps.  First,  Eqs.  (7)  to  (9)  are 

used  to  establish  bounds  for  x,+t,  m,+ |,  %+|,  |x/+1  -  x,+J,  and  ■  — ,  then  Eqs.  (10)  to  (12) 

^/+i  '  O+i 

are  used  to  establish  bounds  on  jc/+l,  v,+!,  tji+i,  and  |x,+i  -  jf/+1|.  The  obvious  epoch  subscripts  are 
dropped  in  the  notation. 

Step  1: 

By  the  triangle  inequality  and  the  assumed  bounds, 

|x*|  -  1/3?  +  «|  <  y  |x|  +  |«i  <  0.52 ln(l/ A)  <  ln(l/A). 

Since  —  €  (0, 11  and  (1  +  *l>x)  €  (0.96,  1.04),  it  likewise  follows  from  Eq.  (9)  that 
in 

lo’l  <  -rlw  +  -T4-  »  4l 


<  \  y/b  +  -&  h 
8  va 


<  3 yfbh,  since  a  -  and  b  -  15. 
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Since  lx,  -  x  |  <  Vln(l//i)  by  assumption,  it  follows  from  Eqs.  (1)  and  (7)  that 
Ixi+i  -  X* |  <  y| x,  -  x|  +  |1  +  *x|M 


<  y  +  (1.04)(0.02)|Vln(l//r)  <  0.52Wln(l/b), 


and  therefore  that 


I*  -  X*|  <  lx,  -  x*|  +  Ivl  <  0.S4iViHn7AT 


From  Eqs.  (3)  and  (8),  it  follows  that 

fli  —  p  —  y  (v  —  p)  +  2^x  +  #2x2. 


Since  w  -  p  is  in  the  interval  la  -  1,  b  -  yl  C  hi,  bl  and 


2*x  +  <b2x2  €  [— 2d>x,  2+x  +  *2x2l  C  -y,  y 


by  the  inequalities  assumed. 


m-  n-a  +  /3,  where  a  €  -y,  y  and/J  €  -y,  y  . 


(m-p)  €  -y,  -~ 


Since p  €  |l,  yj,  m  €  11/4,  6]  C  la,  b)  for  a  -  yy  and  b  -  15. 


Step  2: 

From  Eq.  (12),  hjl  <  |ij*|  <  3>/bb. 


From  Eq.  (11), 


v  -  — i  — ^51 —  (,  _  v») 

m+1  (m  +  l)2  u 


■Jm 

Setting  the  derivative  to  zero  shows  that  - — — rr  is  maximized  when  m  -  1/3,  so 


W.W,  W1LLMAN 


Also,  since  h  <  0.1,  and  h  ln(l /h)  <  0.04, 

i)*(z  -  x •)  <  Ubh  (0.541K//r  ln(l /h) 
<  3VO  (0.541)VO4  <  1. 


Hence, 


!+•») 


.  yfm 
(m  +  l)2 


2 


(r  -  x*) 


€  [1/3.  2). 


Since  m  €  [1/4,6], 


and 


m  ,  _1_  j6 

m+ 1  5' 7 


v  € 


C  la. 6]. 


In  Eq.  (10), 


Since  h  <  0.007, 


and 


m 


m+\ 


U  -  x*|  <  |yj(0.541)Vln(l/A)  <  0.46Wln(l//i). 


J  (:  ~  x*)2 

"1  m+1 


jln(l/A)  >  1, 


-h.  jh  ln(I/A)  —  h\ 


C  [-0.007,0.007], 


_3/2  ^/3 

Also,  - - ttt  is  maximized  when  m  -  3,  giving  -77-.  Hence, 

(m  +  Ir  10 


t)*  7  W— -,  [— — -  l|  €  [-0.035,0.035]. 
(m  +  l)2  m+ 1 


Therefore,  from  the  triangle  inequality  and  Eq.  (10), 

|x  —  x*|  <  0.463  Vln(l//i)  +  0.035  <  Vln(l//t)  for  h  ^ e~s , 


|x |  <  |x*|  +  |x  -  x*|  <  0.983>/ln(l /h)  +  0.035  <  Vln(l/A)  for  h  <  e~s. 


and 

|x,+,  -  x|  <  |x,+l  -  x*|  +  lx*  -  x|  <  0.984  Vln(l/A)  +  0.035  <  Vln(l/Jr)  for  h  <  e~\ 


The  desired  result  follows  by  induction  on  i,  because  the  induction  hypothesis  holds  by  definition 
for  /  -  1. 
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Appendix  C 

LIPSCHITZ  CONDITIONS 


If 

•  I/) Or)  -  ft(y)\  <  A,  +  B,\x  -  >|;  x  and  y  €/?,;  /  -  1  «  ; 

•  IpOcj . xn . xn)  -  g(xj  ,  ....  xi2  ,  ....  x„)|  <  C,  +  Aki  ~  *,2I  ; 

Xu  and xi2  €  ./)(/?,);  xk€/k (Rk),  k  *  t ; 

and 

•  Mx)  A  tfl/iGr),  ....  /,(*)];  x€  n  /?,  ; 

then 

A(x)  -  h(y)  -  gl/i(x),  ....  /ff(x)I  -  st/iCy),  f2(x),  ...  ,  /„(*)] 

+  g[f\  (v).  /2(*).  ....  /« (*)]  -  stA  (y)./2(y) . /»(*)! 


+  sl/i(y) . /n-iCy).  fn(x)1  -  yt/"i(y) . 


By  the  triangle  inequality  and  the  Lipschitz  conditions  for  g : 

|A(x)  -  /r(y)|  <  £  (C,  +  D,\f,(x)  -  /i(y)l). 


/-i 


By  the  Lipschitz  conditions  on  the  /,: 

|A(x)  -  A(y)|  <  £  C,  +  DM  +  B,\x  ->|)-£  (C,  +  A^,>  +  (£  A  A>U  -  j|. 


(-1 


(-1 


/-1 


Furthermore,  by  the  mean  value  theorem: 

•  If  //  is  continuous  on  Rt  -  (a,,AJ,  then  the  above  Lipschitz  condition  on  /,  obtains  with  A,  -  0 
and  B,  -  max  {U'(x)|}. 

xtR, 

•  If  is  continuous  on  /  ([a,,Al),  then  the  above  Lipschitz  conditions  on  g  obtain  with  C,  - 

0  and  A  -  max  |  ~g}Xl)  I 

x(€/((m(d  (  dxt  y 

Specific  Case  (Product)  : 

f\ (x)f2(x)  -  /i(y)/2(y)  -  fi(x)l/2(x)  -  f2(y)]  +  f2(y)[fi(x)  -  /i(y)]. 

so 

\AWf2(x)  -  /i(y)/2(y)|  <  G42!/i0c)I  +  ^tl/2(y)l)  +  (*i  l/tOc)l  +  B\  |/2(y)|)|x  -  y\. 


Appendix  D 

EXISTENCE  AND  APPROXIMATION  OF  RADON-NIKODYM  DENSITY 
Let  R  denote  the  ring  of  all  subsets  of  real  numbers  which  have  the  form 

U  [a„b,). 

/-i 

where  n  is  an  integer,  b,  -  a,  <  a  positive  constant  c,  and 

i  ^  J+  [a,,b,)  n  [a]t  bj)  -  (the  empty  set). 

Consider  the  following  two  measures  on  R: 

•  the  restricted  Lebesgue  measure  /9,  so /8 {[«/,  b, ))  —  bt-  at\  and 

•  the  measure  n  defined  by 

/*(! a,.bi)}  -  Fs[rtan[bhL  -  l),L]  -  £,[max{tf/(  1  -  L],L\ 

for  £,(•,£)  as  in  Section  4.3.  Hence  there  exists  a  k  >  0  such  that  #*{(<»/. 6,))  <  k(b,  -  a,) 
for  all  [a„b,)  of  the  form  described  above. 

By  the  additivity  of  measures  and  the  distributive  law  for  R  (under  set  union  and  intersection), 
M  <  kfi  on  all  of  R.  Also,  it  is  a  standard  result  of  measure  theory  that  both  measures  can  be 
extended  uniquely  to  the  class  S(R)  of  all  Borel  sets,  which  extensions  we  denote  by  4  and  jk. 

For  any  £  €  S(R),  there  is  clearly  a  collection  of  disjoint  intervals  A,  such  that 

£  C  U  A, 

/-l 

and 

A,  €  R,  /  -  1,  2, . 

Since  n  is  induced  by  a  probability  measure,  it  is  finite,  so 

oo  oo 

«(U  A/l  -  pt(At)  (since  the  A,  are  disjoint) 

/-i 

oo 

<  £  kp(A,)  (by  assumption) 

(-1 

-*T/8C4,)-*!§lU  A,]. 
i-i 

This  must  be  true  for  the  infimum  of  all  such  covers  of  £,  so  by  the  standard  construction  of  the  exten¬ 
sions  fi,  and  4, 

A  <  *4  on  all  of  S(R). 

By  the  construction  of  the  standard  completions  of  jk  and  4  on  the  class  S(R  )  of  Lebesgue  measureable 
sets,  therefore, 


H  <  k  A  on  all  of  S(R), 
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where  ji  denotes  the  completion  of  u  and  X  denotes  Lebesgue  measure  (the  completion  of  j§).  This 
means  that  jZ  is  absolutely  continuous  with  respect  to  Lebesgue  measure.  Thus,  by  the  Radon- 
Nikodym  theorem,  the  definition  of  probability  distribution  functions,  andjhe  standard  construction  of 
M  and  ft  from  the  quasi-distribution  function  £,(-,Z),  the  function  Fs(x,L)  has  a  measurable  Radon- 
Nikodym  derivative  with  respect  to  Lebesgue  measure  on  jc  for  1  -  L  <  x  <  L  -  1.  The  same  argu¬ 
ment  also  holds  with  Fs(x,L)  in  place  of  Fs(x,L )  everywhere. 


Now,  in  the  context  of  Eq.  (22)  and  (24)  of  Section  4.3,  let 

e-yPrll/l  >  £) 

and  let  /(ft)  denote  some  Radon-Nikodym  derivative  of  Ft(.8,L).  Hence,  for  all  A  >  0  and  all 
9  €  (1  -  L,L  -  1  -  A), 

J»#+& 

#  fdk  <  cA. 

Since  /  is  measureable,  we  can  define 

£  -  {0  €  (1  -  L,L  -  1):  /(ft)  >  2c) 

and 


m  -  X(£). 


From  the  construction  of  Lebesgue  measure,  for  every  ft  >  0  there  exists  a  disjoint  sequence  of  inter¬ 
vals  lay,  by)  -  F,  such  that 


and 


B  -  U  le„by)  3  £ 


X  (£)  <  m  +ft. 

Therefore,  integration  with  respect  to  Lebesgue  measure  gives 

/#/"J*/  +  /»-£/*  *m+iUB-E)  >  2cm 
and 

/  /-£[/,../]<  X  c(b,  -  a,)  «C  e(m  +  8). 
<-t 1  '  1  <-t 

Hence,  eft  >  c m  for  all  ft  >  0.  Since  c  >  0,  m  -  X(£)  -  0. 


Now  let  <?,-{«  €  (1  -  L.L  -  1 ):/(«)  <  -ft)  for  any  8  >  0.  Then 
-8X(Ge)  >  fG  fdk  -  Pr(s  €  <7,  and  |f  I  >  L)  >  0, 
so  X  (G#)  -  0.  Combining  results,  we  get 

x{ft  €  (1  -L,  L-  1):  /(ft)  >  2e  or /(ft)  <  -ft)  -  0 


for  every  8  >  0.  Finally, 

{ft€  (l-L,  L-  1): /(ft)  €  [0,2 el) 


00 

u 

*>1 


ft  €  (1  -  L,  L  -  1):  /(ft)  e 
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and 


x{fl  €  (1  -  L,  L  -  1 ):/(«)  e[0, 2«]J  £  A{0  €  (1  -  L,  L  -  1):  f(9)  * 

h>  1  l 

Since  Lebesgue  measure  is  positive, 

0  </(#)<€-  Pr{ I f  |  >  L\  for  ail  9  €  (1  -  L,  L  -  1), 


— 2 c 
h’ 


-  0. 


except  perhaps  for  a  set  A  of  Lebesgue  measure  zero,  in  which  case  another  Radon-Nikodym  derivative 
g  can  be  constructed  as 


g(9)  - 


f(9)  if 99  A 
0  if  9  €  A, 


which  satisfies  these  inequalities  for  all  9  €  (1  —  L,  L  -  1). 


Appendix  E 

SOME  INEQUALITIES 

For  x,  k,  a  >  0, 

(x  +  a)"  <  ltmx{a,kn)n e  *  ]e*;  n  —  1,  2,  ... .  (El) 


This  is  established  by  noting  that  (x  +  a)n  e  k  has  the  x  derivative 

x  +  q| 

*  I 


(x  +  a)n  1  e  k 


for  all  positive  x.  This  derivative  is  positive  if  kn  >  x  +  a  and  negative  if  kn  <  x  +  a.  Hence  the 

maximum  value  of  (x  +  a)"e  *  for  x  ^  0  occurs  at  x  -  max{0,  kn  -  a).  Substituting  this  value  for 
x  gives 


(x  +  a)"e-x/k  < 


( kn)"e * 
a ■ 


if  kn  ^  a 
if  k  n  <  a. 


which  is  equivalent  to  the  desired  inequality. 


For  t,  a  ^  0, 

because 


(r  +  o)"<  2"(/"  +  a"), 

(r  +  a)"  <  (2max{f,  a)]"  <  2',max{t",  an J  <  2*(i"  +  a"). 


(E2) 
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